Comet: Decentralized Complex Event Detection in Mobile Delay Tolerant Networks

Abstract

Abstract-Increased commodity use of mobile devices with sensory capabilities has the potential to enable mission-critical monitoring applications in various domains. However, these mobile-enabled monitoring applications have to often work in environments where a delay-tolerant network (DTN) is the only feasible communication paradigm. DTNs are multi-hop networks prone to long delays and frequent disruptions. Unfortunately, there is a lack of effective and scalable support for building sophisticated DTN-based monitoring applications. Detection of complex (composite) events, for example, is fundamental to monitoring applications. However, there is little work in designing scalable complex event detection (CED) techniques for DTNs. The current plan-based CED techniques are mostly centralized, and hence are inherently unscalable for DTNs.In this paper, we create Comet, -a decentralized plan-based, efficient and scalable CED for DTNs. Comet shares the task of detecting complex events (CEs) among multiple nodes, with each node detecting a part of the CE by aggregating two or more primitive events or sub-CEs. Comet uses a unique h-function to construct cost and delay efficient CED trees. As finding an optimal CED plan requires exponential-time, Comet finds near-optimal detection plans for individual CEs through two novel heuristic planning techniques: multi-level push-pull conversion and virtual CED tree creation. Performance results show that Comet reduces cost by up to 89% compared to pushing all primitive events and over 60% compared to a two-level exhaustive search algorithm.

Network-wide complex event processing over geographically distributed data sources

“…What is more, each pair of (push-pull strategy, in-network placement) for a given operator affects the push-pull and placement choices not only of all directly connected operators, but also of all EDG paths that pass through that given operator. Some of the efforts outlined in this section attempt to tackle specific aspects of the problem, such as [A2] [4] or restrict [A] to a single operator instead of an EDG and its aforementioned paths [36]. Hence, they do not confront the greater challenge of the generic problem setup addressed by our work, as described above.…”

“…Geographically Distributed CEP [64]: There is a pair of works that are more closely related to ours. The first is the work of Comet [36], which proposes to combine in-network operator placement with a push-pull mechanism for communication-efficient and latency-aware CE detection over mobile networks. Nonetheless, the approach is restricted to CEP queries with only one operator per query.…”

“…As a result of the development of a streaming multi-cloud platform that uses our algorithms [34,35], our work elaborates on real system implementation aspects and enhances its contributions along these lines. On the contrary, prior techniques such as [4,36] may, for instance, opt for [A2] but because of lacking [C] they do not comment on architectural aspects or provide implementation hints over real networks, as we do. Contributions.…”

“…In Section 4 we outline the statistics that are necessary to our optimization algorithms for devising a preferable plan for executing the posed geo-distributed CEP analytics tasks and provide the formal Push-pull Enhanced CEP Operator Placement (PECOP) problem definition. We introduce Dynamic Programming (DP) and Exhaustive Search (ES) algorithms that provide Feature Category Algorithmic Service System Related Work A1 A2 B1 B2 C Akdere et al [4] Cardellini et al [38], SODA [39] SBON [37], Hermes [40], DHCEP [41], SAND [42], SPADE [43] Cordies [44], PADRES [45] Kumar et al [46], Rizou [47] Iridium [48], JetStream [49] Borealis [50], Medusa [51], Chatzistergiou et al [52], Gu et al [53], Flux [54], Zhou et al [55] SQPR [56], Geode [57] Chatzimilioudis et al [58], Srivastava et al [59], Ying et al [60] Amini et al [61], Benzing et al [62] Repantis et al [63] Comet [36] This work optimal solutions to PECOP in the absence (DP) or existence (ES) of events that are shared among CEP operators, in Section 5. Besides the computationally expensive DP and ES, Section 5 presents greedy and heuristic algorithms for quickly deriving efficient solutions to PECOP in practical scenarios.…”