FlinkCEP is the Complex Event Processing (CEP) API of the Flink Big Data platform. The high expressive power of the language of FlinkCEP comes at the cost of cumbersome parameterization of the queried patterns, acting as a barrier for FlinkCEP's adoption. Moreover, properly configuring a FlinkCEP program to run over a computer cluster requires advanced skills on modern hardware administration which non-expert programmers do not possess. In this work (i) we build a novel, logical CEP operator that receives CEP pattern queries in the form of extended regular expressions and seamlessly re-writes them to FlinkCEP programs, (ii) we build a CEP Optimizer that automatically decides good job configurations for these FlinkCEP programs. We also present an experimental evaluation which demonstrates the significant benefits of our approach.
Enabling real-time processing of financial data streams is extremely challenging, especially considering that typical operations that interest investors often require combining data across (a potentially quadratic number of) different pairs of stocks. In this paper we present the architecture and the components of our system for the real-time processing of geo-distributed financial data at scale. Our system can scale to larger resources and utilizes a Synopses Data Engine in order to efficiently handle complex cross-stock queries, such as the ones required to detect systemic risk or to help forecast the value of some stock. The rich set of supported operations is depicted at the Visual Analytics component of our system.
CCS CONCEPTS• Computer systems organization → Real-time systems.
We present a Maritime Situational Awareness (MSA) framework for detecting and forecasting maritime events (e.g., illegal fishing) over streams of Big maritime Data. The architecture of the MSA framework relies on the following state-of-the-art components: (i) the Maritime Event Detector which uses data-driven distributed techniques deployed on a computer cluster to detect maritime events of interest in an online, real-time fashion, (ii) the Complex Event Forecasting module, which implements state-of-the-art distributed Complex Event Forecasting techniques for maritime data, (iii) the Synopses Data Engine component, that creates synopses of maritime data improving the scalability of the framework and (iv) the streaming extension of a popular data science platform, namely RapidMiner Studio, that integrates all the above, allowing users to graphically design and rapidly implement Big Data analytics pipelines which can be deployed transparently on top of distributed architectures.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.