Maximizing Determinism in Stream Processing Under Latency Constraints

Zacheilas, Nikos; Kalogeraki, Vana; Nikolakopoulos, Yiannis; Gulisano, Vincenzo; Papatriantafilou, Marina; Tsigas, Philippas

doi:10.1145/3093742.3093921

Cited by 27 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assumed a hierarchical organization of edge nodes; multiple event sinks within non-hierarchical systems [28] can achieve highly decentralized systems and compositional applications. Moreover, we ignored management of outof-order events [29] and their effect [30], rendering our approach opportunistic. Furthermore, hyperproperties relating multiple computation traces with each other have been theoretically investigated; in such properties, it is necessary to store previously seen traces, and to relate new traces to the traces seen so far.…”

Section: Complex Event Processing and Data Streamsmentioning

confidence: 99%

Edge-Based Runtime Verification for the Internet of Things

Tsigkanos

Bersani

Frangoudis

et al. 2022

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

Complex distributed systems such as the ones induced by Internet of Things (IoT) deployments, are expected to operate in compliance to their requirements. This can be checked by inspecting events flowing throughout the system, typically originating from end-devices and reflecting arbitrary actions, changes in state or sensing. Such events typically reflect the behavior of the overall IoT system -they may indicate executions which satisfy or violate its requirements. This paper presents a service-based software architecture and technical framework supporting runtime verification for widely deployed, volatile IoT systems. At the lowest level, systems we consider are comprised of resource-constrained devices connected over wide area networks generating events. In our approach, monitors are deployed on edge components, receiving events originating from end-devices or other edge nodes. Temporal logic properties expressing desired requirements are then evaluated on each edge monitor in a runtime fashion. The system exhibits decentralization since evaluation occurs locally on edge nodes, and verdicts possibly affecting satisfaction of properties on other edge nodes are propagated accordingly. This reduces dependence on cloud infrastructures for IoT data collection and centralized processing. We illustrate how specification and runtime verification can be achieved in practice on a characteristic case study of smart parking. Finally, we demonstrate the feasibility of our design over a testbed instantiation, whereupon we evaluate performance and capacity limits of different hardware classes under monitoring workloads of varying intensity using state-of-the-art LPWAN technology.

show abstract

Section: Complex Event Processing and Data Streamsmentioning

confidence: 99%

Edge-Based Runtime Verification for the Internet of Things

Tsigkanos

Bersani

Frangoudis

et al. 2022

IEEE Trans. Serv. Comput.

View full text Add to dashboard Cite

show abstract

“…Stream Processing in a nutshell. Stream processing is leveraged in large distributed systems [1,2,7,9,10,13,22,23] to process unbounded streams of tuples. Stream processing applications are defined as Directed Acyclic Graphs of operators that transform the tuples delivered by a set of data sources and produce new streams of tuples that are eventually delivered to end-users.…”

Section: Talk Overviewmentioning

confidence: 99%

Motivations and Challenges for Stream Processing in Edge Computing

Gulisano

2021

Companion of the ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

The 2030 Agenda for Sustainable Development of the United Nations General Assembly defines 17 development goals to be met for a sustainable future. Goals such as Industry, Innovation and Infrastructure and Sustainable Cities and Communities depend on digital systems. As a matter of fact, billions of Euros are invested into digital transformation within the European Union, and many researchers are actively working to push state-of-the-art boundaries for techniques/tools able to extract value and insights from the large amounts of raw data sensed in digital systems.Edge computing aims at supporting such data-to-value transformation. In digital systems that traditionally rely on central data gathering, edge computing proposes to push the analysis towards the devices and data sources, thus leveraging the large cumulative computational power found in modern distributed systems. Some of the ideas promoted in edge computing are not new, though. Continuous and distributed data analysis paradigms such as stream processing have argued about the need for smart distributed analysis for basically 20 years. Starting from this observation, this talk covers a set of standing challenges for smart, distributed, and continuous stream processing in edge computing, with real-world examples and use-cases from smart grids and vehicular networks.

show abstract

“…Most recently, reference [31] utilizes workload estimation from [32] together with operator-profiling for parallelizing. Reference [33] handles unordered arrival of multiple unsynchronized input sources by newly defining slack-ready tuple to provide a deterministic solution while keeping real-time requirement satisfaction. Reference [34] maps workload partitioning and scheduling in clustered Storm [11] which is stream-based engine into the graph-partitioning problem to come through the high performance of resource utilization which reducing network loads.…”

Section: Task Distribution Provision On Cepmentioning

confidence: 99%

A Self-Organized Task Distribution Framework for Module-Based Event Stream Processing

2019

View full text Add to dashboard Cite

Tackling bottleneck and privacy issues of cloud computing, we attempt to push event stream processing down to devices which are currently empowered to compute and communicate at the edge of the networks. To accomplish that, we propose a self-organized task distribution framework that is composed of multiple brokers collaborating through our module-based event stream processing engine called EdgeCEP. Our system request is event-dependent specified in a brand-new event specification language; still, the event is stored and processed by the relational database. We newly formulate the problem of self-organized task distribution subjective to preferable constraints of computation and communication. The solution for each broker to find individual optimal decision is to apply tabu search with flow-based greedy move regarding pre-ranking flow table. Many experiments are conducted to study and evaluate the performance of the proposed system. The simulation shows that the proposed flow optimization outperforms the naïve algorithm, concretely, 2-times more tasks getting processed and successfully delivered within the same fixed period. The proposed edge-centric method achieves data traffic 7-times less than the cloud-centric approach. The prototype engines have been deployed and evaluated in the real environment. INDEX TERMS Stream processing, complex event processing, edge computing, self-organized task distribution, Internet of Things (IoT).

show abstract

Maximizing Determinism in Stream Processing Under Latency Constraints

Cited by 27 publications

References 18 publications

Edge-Based Runtime Verification for the Internet of Things

Edge-Based Runtime Verification for the Internet of Things

Motivations and Challenges for Stream Processing in Edge Computing

A Self-Organized Task Distribution Framework for Module-Based Event Stream Processing

Contact Info

Product

Resources

About