Complex Event Processing (CEP) is of increasing importance in many industrial applications to integrate a huge number of events in a scalable manner. A core challenge towards scalable CEP is to efficiently distribute the rules which define how correlations between events can be detected within an event processing network. Although significant progress has been made recently, there remains a fundamental gap in supporting requirements that emerge from deploying CEP over heterogeneous and independent processing environments. Heterogeneity typically imposes many constraints on the placement of rules, which increases the complexity of the underlying optimization problem and cannot be handled efficiently by existing solutions. In this paper we examine the distributed placement, migration and optimization of rules in the context of the constraint optimization problem to minimize network usage. We propose and evaluate a placement algorithm that efficiently finds valid solutions in scenarios where the solution space is heavily restricted by constraints. The algorithm operates in a decentralized way and is adaptive to dynamic changes of processing nodes, rules, and load characteristics of the event processing network. The proposed rule migration policies resolve invalid placements quickly and thus ensure high availability. The evaluations show that the proposed algorithm is able to efficiently find near optimum solutions within heavy constraint-driven network conditions.
Although a significant amount of research has investigated the benefits of distributed CEP in terms of scalability and extensibility, there is an ongoing reluctance in deploying distributed CEP in an industrial context. In this paper we present the DHEP system developed together with the IBM R 1 laboratory in Böblingen. It addresses some of the key problems in increasing the acceptance of distributed CEP, for example supporting interoperability between heterogeneous event processing systems. We present the concepts behind the DHEP system and show how those concepts help to achieve scalable and extensible event processing in an industrial context. Moreover, we verify in an evaluation study that the additional cost imposed by the DHEP system is moderate and 'affordable' for the benefits provided.
Complex event processing (CEP) is an emerging software technology for detecting business-relevant situations in streams of events and for providing these detected situations to business processes that are interested in it. While currently CEP-systems are mostly deployed within a single business domain at a limited scale, the cooperative nature of business applications gives reason that complex event processing will soon address multiple business domains and involve an increasingly large number of business events. In order to ensure interoperability as well as efficient utilization of processing and network capability, we motivate the need for heterogeneous correlation technology in the context of business applications. In this article we give an overview of the project “Distributed heterogeneous event processing” (DHEP) involving the IBM Böblingen lab and the Universität Stuttgart, where this need is adressed. The key aspect of the project is the deployment of collections of event correlation rules to a network of heterogeneous event correlation engines. We give an overview of challenges and possible solutions for the dynamic configuration of such environments and present our architecture which supports network-wide cooperation between different correlation engines.
Current event processing systems lack methods to preserve privacy constraints of incoming event streams in a chain of subsequently applied stream operations. This is a problem in large-scale distributed applications like a logistic chain where event processing operators may be spread over multiple security domains. An adversary can infer from legally received outgoing event streams confidential input streams of the event processing system. This paper presents a finegrained access management for complex event processing. Each incoming event stream can be protected by the specification of an access policy and is enforced by algorithms for access consolidation. The utility of the event processing system is increased by providing and computing in a scalable manner a measure for the obfuscation of event streams. An obfuscation threshold as part of the access policy allows to ignore access requirements and deliver events which have achieved a sufficient high obfuscation level.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.