Parallel event processing on unbound streams with multi-step windowing

Abstract

We describe an approach for a custom complex event processing engine using Message Passing Interface (MPI) in C++ programming language. Our approach utilizes a multi-processor infrastructure and distributes its load on multiple processes, expecting each process to run on one processor. A dispatching process receives events and distributes them on several query processes which are responsible for updating the actual queries. Query processes forwards any updates to a presentation process that output the results in an appropriate format. The distribution of roles among processes allows better scalability since further query processes can be added dynamically to handle more queries. In our evaluation we measured event processing up to a throughput of 12k events/sec using 4 processor cores.