View source

From BiCEP

== Overview == * Retrieve, interpret and store data source events * Predefined and custom data sources * Visualizations * Queries * Alerts * Tags * Web and mobile-based frontends * History == State of the Art == === Alerts.com === The information you want when and where you want it. You're in control. Weather, news, traffic, gas prices, reminders, live sports and more. It's new, it's free, your personal information is secure, no spam. [http://www.alerts.com/ www.alerts.com] === Chartlet === Chartlet is a free Firefox extension that periodically checks places on the Web, extract and stores numerical values. On demand, you can display the history of those values in nice charts. [http://www.chartlet.net/ www.chartlet.net] === WebSundew === WebSundew is an advanced web scraping tool that extracts data from web pages on the internet: 100% graphic user interface, step-by-step template wizards, automatic iteration for linked pages, macros for automatic log into sites and data extraction. [http://www.websundew.com/ www.websundew.com] === RRDtool === RRDtool is the OpenSource industry standard, high performance data logging and graphing system for time series data. Use it to write your custom monitoring shell scripts or create whole applications using its Perl, Python, Ruby, TCL or PHP bindings. [http://oss.oetiker.ch/rrdtool/ oss.oetiker.ch/rrdtool] == Data Sources == === Predefined === * TBD Sources: stocks, flight schedules, weather, sports, auctions, traffic reports, feeds, sensors, ... * Makes use of data source API * May make use of external APIs === Custom === * TBD Sources: html table, html items, ... * Makes use of data source API * User may identify items to monitor in the following ways: ** '''Single Page Entry''' -- single page items ** '''Same Level Page Entries''' -- every news title, repetitive patterns ** '''Table Selection''' -- smartly select all table items ** '''Table Line Selection by Constant Column Value''' -- select something in the same row/column whose column/row has something wrote on it ** '''Automatic Selection Inside Area''' -- smartly select every item inside a bounded polygon or even the whole page ** '''Advanced Text Selection''' -- left of something and/or right of something, bounded by something * Browser extension is an option * Each item data type must be automatically chosen by the system. May be overridden by the user. == Data Source API == * Modular approach * Every DS adapter will make use of this API * Each item has the following properties: ** '''Name''' -- identification ** '''XPath''' -- location in the page ** '''Type''' -- data type (following section) == Data Types == * Suggest the best type for each item * TBD Data types: ** Integer ** Decimal ** Percentage ** Value -- temp, speed, height, weight, ... ** Time ** Date ** DateTime -- event timestamp ** String * Connect similar/equivalent data types: ** Celsius, Fahrenheit, Kelvin, ... ** Metre, Mile, Yard, Foot, ... ** Euro, U.S. Dollar, Pound, Swiss franc, ... == Data Interpretation == * Show data from several angles TBD * Mix/Join data sources == Data Store == * Performance * Scalability * Use RRDtool or a similar approach ** Round Robin Database ** Bounded storage footprint ** Sampling ** Custom levels of aggregation * Client-side history * Queries, visualisations and filters may be stored for future re-utilization by any user * '''Distributed memory (condor-like)''' == User Interface == * Web-based * Mobile * TBD Push/Pull-based == Miscellaneous == * OpenID login * i18n == Firefox Extension == [http://student.dei.uc.pt/~rjmm/firefox/9ticks.xpi 9ticks Firefox extension] === Use cases === * [http://eztv.it/index.php?main=airs Series Countdown List] * [http://dsg.dei.uc.pt/ganglia/ DSG Cluster] * [http://www.maisfutebol.iol.pt/superliga_classificacao.php Liga Sagres] * [http://news.bbc.co.uk/sport2/hi/football/eng_prem/table/default.stm Barclays Premier League] * [http://weather.no.sapo.pt/details.html Manteigas Weather] * [http://www.osnews.com/ OSNews] * [http://finance.google.com/finance?q=NASDAQ:AAPL Apple Google Finance] * [http://www.timeanddate.com/counters/newyear.html New Year Countdown] * [http://www.traffic.com/New-York-Traffic/I-278_Gowanus_Expressway___Brooklyn_Queens_Exwy-NORTHBOUND-7_NB_GW_BQE_12.html NY I-278 Traffic] * [http://www.ebay.com/ Ebay]

Return to 9ticks:Older.