DSPBench: A Suite of Benchmark Applications for Distributed Data Stream Processing Systems

Bordin, Maycon Viana; Griebler, Dalvan; Mencagli, Gabriele; Geyer, Cláudio F. R.; Fernandes, Luiz Gustavo

doi:10.1109/access.2020.3043948

Cited by 26 publications

(23 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) Word count topology [18]: The spout emits streams of sentences and sends them to a bolt that splits these sentences into words and emits them to another bolt that, using field grouping, can count how many times each word has occurred. Field grouping means that based on the value of the word, the same word must always go to the same instance so that it can be counted.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…4) Spike detection topology [18]: The spout receives a stream of data from sensors and emits them to bolts to monitor the occurrences of values that have spikes. Spout emits this stream to a bolt named Moving Average, which gets the data grouped according to the IDs of the device.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…2) Complete Latency: is the average time a tuple takes to be entirely processed by the topology [18].…”

Section: B Results and Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Balanced Schedule on Storm for Performance Enhancement

Selim¹,

El-Attar²,

Hanafy³

et al. 2022

IJACSA

View full text Add to dashboard Cite

In recent years, real-time and big data aroused and received a lot of attention due to the spread of embedded systems in almost everything in life. This has led to many challenges that need to be solved to enhance and improve systems that work on big real-time data. Apache Storm is a system used for computing and analyzing big real-time data of distributed systems. This paper aims to develop a scheduler to improve the scheduling of the applications represented by topologies on the Storm cluster. The proposed scheduler is hybridization between the scheduling algorithms of A3 Storm and the Workload scheduler. Its objective is to minimize the communication between tasks while balancing the workload on all cluster machines. The proposed scheduler is compared with the A3 Storm and Fischer and Bernstein's scheduling algorithm. The comparison has been made using four different topologies. The experimental results show that our proposed scheduler outperforms the two other schedulers in throughput and complete latency.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Experiments and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Balanced Schedule on Storm for Performance Enhancement

Selim¹,

El-Attar²,

Hanafy³

et al. 2022

IJACSA

View full text Add to dashboard Cite

show abstract

“…We analyze seven applications 5 used in the literature [2] and whose DAGs are in Figure 11. FraudDetection (FD) applies a Markov model [26] to calculate the probability of a credit card transaction being a fraud.…”

Section: Applicationsmentioning

confidence: 99%

“…However, recent SPSs have extended the scope of their supported applications to go beyond the domain of relational algebra. This is done by dealing with both structured and unstructured data, and supporting the execution of complex computational tasks, even adding the possibility to leverage external tools for specific domains (e.g., TensorFlow and PyTorch for Machine Learning [2]).…”

Section: Introductionmentioning

confidence: 99%

WindFlow: High-Speed Continuous Stream Processing With Parallel Building Blocks

Mencagli

Torquati

Cardaci

et al. 2021

IEEE Trans. Parallel Distrib. Syst.

Self Cite

View full text Add to dashboard Cite

Nowadays, we are witnessing the diffusion of Stream Processing Systems (SPSs) able to analyze data streams in near realtime. Traditional SPSs like STORM and FLINK target distributed clusters and adopt the continuous streaming model, where inputs are processed as soon as they are available while outputs are continuously emitted. Recently, there has been a great focus on SPSs for scale-up machines. Some of them (e.g., BRISKSTREAM) still use the continuous model to achieve low latency. Others optimize throughput with batching approaches that are, however, often inadequate to minimize latency for live-streaming applications. Our contribution is to show a novel software engineering approach to design the runtime system of SPSs targeting multicores, with the aim of providing a uniform solution able to optimize throughput and latency. The approach has a formal nature based on the assembly of components called building blocks, whose composition allows optimizations to be easily expressed in a compositional manner. We use this methodology to build a new SPS called WINDFLOW. Our evaluation showcases the benefits of WINDFLOW: it provides lower latency than SPSs for continuous streaming, and can be configured to optimize throughput, to perform similarly and even better than batch-based scale-up SPSs.

show abstract