L-BiX: incremental sliding-window aggregation over data streams using linear bidirectional aggregating indexes

Bou, Savong; Kitagawa, Hiroyuki; Amagasa, Toshiyuki

doi:10.1007/s10115-020-01444-5

Cited by 9 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FIFO algorithms have been extensively studied to achieve an O(1) time complexity. They can be categorized into: (1) worst-O(1) (e.g., L-BiX [11] and Two-Stacks [19]) and (2) average-O(1) (e.g., FlatFIT [21], DABA [19], and FOA&IOA [19]). L-BiX [11] is currently the most efficient FIFO algorithm.…”

Section: Fifo Algorithmsmentioning

confidence: 99%

“…They can be categorized into: (1) worst-O(1) (e.g., L-BiX [11] and Two-Stacks [19]) and (2) average-O(1) (e.g., FlatFIT [21], DABA [19], and FOA&IOA [19]). L-BiX [11] is currently the most efficient FIFO algorithm. It uses a single array to maintain the aggregating values from the previous computations.…”

Section: Fifo Algorithmsmentioning

confidence: 99%

“…All algorithms were implemented using Java. Note, the implementations of all FIFO algorithms and FlatFAT were from our L-BiX paper [11].…”

Section: Experimental Environmentmentioning

confidence: 99%

See 2 more Smart Citations

CPiX: Real-Time Analytics Over Out-of-Order Data Streams by Incremental Sliding-Window Aggregation

Bou

Kitagawa

Amagasa

2022

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

Stream processing is used in various fields. In the field of big data, stream aggregation is a popular processing technique, but it suffers serious setbacks when the order of events (e.g., stream elements) occurring is different from the order of events arriving to the systems. Such data streams are called "non-FIFO steams". This phenomenon usually occurs in a distributed environment due to many factors, such as network disruptions, delays, etc. Many analyzing scenarios require efficient processing of such non-FIFO streams to meet various data processing requirements. This paper proposes an efficient scalable checkpoint-based bidirectional indexing approach, called CP iX, for faster real-time analysis over non-FIFO streams. CPiX maintains the partial aggregation results in an on-demand manner per checkpoint. CPiX needs less time and space than the state-of-the-art approach. Extensive experiments confirm that CPiX can deal with out-of-order streams very efficiently and is, on average, about 3.8 times faster than the state-of-the-art approach while consuming less memory.

show abstract

Section: Fifo Algorithmsmentioning

confidence: 99%

Section: Fifo Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

CPiX: Real-Time Analytics Over Out-of-Order Data Streams by Incremental Sliding-Window Aggregation

Bou

Kitagawa

Amagasa

2022

IEEE Trans. Knowl. Data Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…[51,52] address these different types of aggregation functions. Bou et al [10][11][12] present multiple techniques for out-of-order incremental sliding window aggregation. To give an overview of the different techniques, Hirzel et al [30] survey several sliding window aggregation algorithms and summarize that the choice of the best algorithm depends on the aggregation operation, latency requirements, window type, sharing requirements, and out-oforder processing.…”

Section: Related Workmentioning

confidence: 99%

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

et al. 2022

View full text Add to dashboard Cite

Window aggregations and windowed joins are central operators of modern real-time analytic workloads and significantly impact the performance of stream processing systems.This paper gives an overview of state-of-the-art research in this area conducted by the Berlin Institute for the Foundations of Learning and Data (BIFOLD) and the Technische Universität Berlin. To this end, we present different algorithms for efficiently processing windowed operators and discuss techniques for distributed stream processing. Recently, several approaches have leveraged modern hardware for windowed stream processing, which we will also include in this overview. Additionally, we describe the integration of windowed operators into various stream processing systems and diverse applications that use specialized window operations.

show abstract

InTrans: Fast Incremental Transformer for Time Series Data Prediction

Bou

Amagasa

Kitagawa

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

L-BiX: incremental sliding-window aggregation over data streams using linear bidirectional aggregating indexes

Cited by 9 publications

References 37 publications

CPiX: Real-Time Analytics Over Out-of-Order Data Streams by Incremental Sliding-Window Aggregation

CPiX: Real-Time Analytics Over Out-of-Order Data Streams by Incremental Sliding-Window Aggregation

Algorithms for Windowed Aggregations and Joins on Distributed Stream Processing Systems

InTrans: Fast Incremental Transformer for Time Series Data Prediction

Contact Info

Product

Resources

About