View source

=== Major Components ===

* User Interface
* Crawler
* Data Store
* Predefined Sources
* Custom Source UI
* '''?'''

=== Crawler ===

* Scheduling Policy
* Distributed crawl

=== Data Store ===

* BigTable + GFS
* Hadoop + HBase
* Map-Reduce
* RRDtool

=== Data Store Bigtable-like ===

{| class="TablePager" border="1"
|-
! Row Key
! Timestamp
! Column "property:trade"
! Column "property:volume"
! Column "property:temperature"
! Column "property:humidity"
|-
| stock.AAPL
| t1
| 109.12
| 200
|
|
|-
| 
| t2
| 109.10
| 
|
|
|-
| 
| t3
| 109.05
| 200
|
|
|-
| weather.Coimbra
| t7
| 
| 
| 13
| 87
|-
| 
| t8
| 
| 
| 14
| 78
|}


{| class="TablePager" border="1"
|-
! Row Key
! Timestamp
! Column "property:trade"
! Column "property:volume"
|-
| stock.AAPL
| t1
| 109.12
| 200
|-
| 
| t3
| 109.05
| 200
|}


{| class="TablePager" border="1"
|-
! Row Key
! Timestamp
! Column "property:trade"
|-
| stock.AAPL
| t2
| 109.10
|}


{| class="TablePager" border="1"
|-
! Row Key
! Timestamp
! Column "property:temperature"
! Column "property:humidity"
|-
| weather.Coimbra
| t7
| 13
| 87
|-
| 
| t8
| 14
| 78
|}