In-memory caching for fast stream history access

Abstract

In data stream systems, it is necessary to access the historical stream data. For example, in a CEP system historical data is necessary for deriving complex events where the underlying atomic events are distributed over a potentially long period of time. Historical stream data is also important in trend analysis systems where new or changed analysis criteria have to be checked on past data. The requirements for history access are manifold; it should be as fast as possible and ought to fit homogeneously into the existing system structure. The latter implies that the management of the constantly growing historical data is automatically performed by the system itself and data can be freely accessed using declarative queries. In this paper, we present an approach relying on the combination of a server-based DBMS and a client-based in-memory DBMS for accelerating access to GeoStream history. The recently developed "Airspace Monitoring System" (AIMS), that monitors and analyzes flight data streams in real time, will serve as a testbed for our approach. We show that an efficient and flexible video recorder and time slider functionality can be implemented using the suggested caching approach