An Energy-Efficient Stream Join for the Internet of Things

Michalke, Adrian; Grulich, Philipp M.; Lutz, Clemens; Zeuch, Steffen; Markl, Volker

doi:10.1145/3465998.3466005

Cited by 11 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Users can choose to coarsen the granularity of timestamps, thus causing more cases of equal timestamps, recovering basic batching. However, it would require additional work to take full advantage of batching, such as for energy efficiency [18]. Our implementation does not directly use SIMD instructions, but the C++ optimizing compiler sometimes uses them automatically.…”

Section: Methodsmentioning

confidence: 99%

Out-of-Order Sliding-Window Aggregation with Efficient Bulk Evictions and Insertions

Tangwongsan,

Hirzel,

Schneider

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

Sliding-window aggregation is a foundational stream processing primitive that efficiently summarizes recent data. The state-of-the-art algorithms for sliding-window aggregation are highly efficient when stream data items are evicted or inserted one at a time, even when some of the insertions occur out-of-order. However, real-world streams are often not only out-of-order but also bursty, causing data items to be evicted or inserted in larger bulks. This paper introduces a new algorithm for sliding-window aggregation with bulk eviction and bulk insertion. For the special case of single insert and evict, our algorithm matches the theoretical complexity of the best previous out-of-order algorithms. For the case of bulk evict, our algorithm improves upon the theoretical complexity of the best previous algorithm for that case and also outperforms it in practice. For the case of bulk insert, there are no prior algorithms, and our algorithm improves upon the naive approach of emulating bulk insert with a loop over single inserts, both in theory and in practice. Overall, this paper makes high-performance algorithms for sliding window aggregation more broadly applicable by efficiently handling the ubiquitous cases of out-of-order data and bursts.

show abstract

Section: Methodsmentioning

confidence: 99%

Out-of-Order Sliding-Window Aggregation with Efficient Bulk Evictions and Insertions

Tangwongsan,

Hirzel,

Schneider

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…Since these devices are batterypowered, they have a limited energy budget, which becomes more taxed when processing complex workloads. Michalke et al propose ecoJoin [44], a stream join operator exploiting the modern hardware of these edge devices to reduce energy consumption. In particular, ecoJoin focuses on devices that combine embedded CPUs and GPUs on a single system.…”

Section: Stream Slicingmentioning

confidence: 99%

“…Michalke et al [44] demonstrate that EcoJoin significantly enhances performance with regard to throughput and power consumption compared to state-of-the-art stream join algorithms. Adjusting batch sizes and scaling the clock frequency leads to improved energy efficiency.…”

Section: Algorithms For Optimizations On Modern Hardwarementioning

confidence: 99%

“…The approaches presented in Sect. 4 exploit advantages of modern hardware, such as multi-core processors and high- Cutty [14], Scotty [65,66,68], AJoin [34], Eco-Join [44], Zhang et al [76], Parallel ADWIN [26] Modern Hardware Zeuch et al [72], Grizzly [28], Slash [21] Distributed Window Processing AStream [35], Disco [8], Rhino [6,20] Stream Processing Systems Scotty Connectors to Apache Flink [13], Storm [61], Beam [3], Samza [47], Kafka Streams [38], Spark [60,71]; NebulaStream [73,74], Agora [69], SENSE [62,63,67] Applications…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

et al. 2022

Self Cite

View full text Add to dashboard Cite

Window aggregations and windowed joins are central operators of modern real-time analytic workloads and significantly impact the performance of stream processing systems.This paper gives an overview of state-of-the-art research in this area conducted by the Berlin Institute for the Foundations of Learning and Data (BIFOLD) and the Technische Universität Berlin. To this end, we present different algorithms for efficiently processing windowed operators and discuss techniques for distributed stream processing. Recently, several approaches have leveraged modern hardware for windowed stream processing, which we will also include in this overview. Additionally, we describe the integration of windowed operators into various stream processing systems and diverse applications that use specialized window operations.

show abstract