Main Page

From BiCEP

Contents 1 Description 2 Publications 2.1 Conferences and Journals 2.2 Demos 2.3 Tutorials 3 Software

Description

BiCEP, a project from the Systems and Software Engineering Group at the University of Coimbra, has the goal to study and improve performance and produce benchmarks for Event Processing systems (CEP). In the papers below we present benchmarking experiments that evaluate engines CEP along several metrics. Some of these are obvious, others not quite: throughput, maximum latency, latency degradation ratio, post-peak latency variation ratio, memory consumption, instruction count, clocks-per-instruction, cache miss rates and more. We studied many different basic and not so basic operations such as selection, projections, aggregations, joins, windows, and patterns detection and the impact of types and sizes of windows, query sharing, variations to peaks, the effects of garbage collection options, and internal data structures and tuple representations.

To perform all this, we built FINCoS, a testing framework (available below) that lets users generate and/or load test data, start and play benchmarking experiments, visualize and capture metrics, and run multiple engines at the same time.

The project started in September 2007 and is partially funded by an FP6 Marie Curie International Reintegration Grant.

Publications

Conferences and Journals

• Assessing and Optimizing Microarchitectural Performance of Event Processing Systems (pdf).
  Marcelo R. N. Mendes, Pedro Bizarro, and Paulo Marques.
  In the Second TPC Technology Conference on Performance Evaluation & Benchmarking (TPC TC).
  Collocated with VLDB10, September 17, 2010 - Singapore.
• A Performance Study of Event Processing Systems (pdf).
  Marcelo R. N. Mendes, Pedro Bizarro, and Paulo Marques.
  In the First TPC Technology Conference on Performance Evaluation & Benchmarking (TPC TC).
  Collocated with VLDB09, August 24, 2009 - Lyon France.
• BiCEP - Benchmarking Complex Event Processing Systems (pdf).
  Pedro Bizarro
  in Dagshtul Event Processing Seminar, 2007. (POSITION PAPER)

Demos

• A framework for performance evaluation of complex event processing systems (pdf).
  Marcelo R. N. Mendes, Pedro Bizarro, and Paulo Marques.
  In the Proc. of the Intl. Conf. DEBS 2008: 313-316.

Tutorials

Benchmarking Event Processing Systems: Current State and Future Directions
Marcelo Mendes, Pedro Bizarro, Paulo Marques
In the First Joint WOSP/SIPEW International Conference on Performance Engineering, San Jose, California, USA, January 28, 2010. (pdf)

Software

• FINCoS framework, version 2.2.1, December 2009 (zip)

The FINCoS framework is a Java-based set of benchmarking tools for load generation and performance measuring of Event Processing systems. It provides a flexible and neutral approach for experimenting diverse CEP systems, where load generators, datasets, queries, and adapters can be easily attached, swapped, reconfigured and scaled.