DABS-Storm: A Data-Aware Approach for Elastic Stream Processing

Kotto-Kombi, Roland; Lumineau, Nicolas; Lamarre, Philippe; Rivetti, Nicolò; Busnel, Yann

doi:10.1007/978-3-662-58664-8_3

Cited by 6 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The solution was validated emphasizing improvements in latency and resources consumption. This work was extended in Reference 74 for improving the performance and stability.…”

Section: Results Analysis and Discussionmentioning

confidence: 99%

“…Kombi et al 74 presented DABS‐Storm for elastic stream processing on Storm by dynamically adapting the parallelism degree. Importantly for generalization purposes, they argued that the proposed solution could be implemented is other related solutions.…”

Section: Results Analysis and Discussionmentioning

confidence: 99%

“…Consequently, reactive approaches may eventually violate QoS, but they tend to respond better to fluctuating workloads. References 37,47,63,65,66,74,75,82,98 claimed to be proactive.…”

Section: Adaptation Decisionsmentioning

confidence: 99%

“…RQ6 and the category described in Section 4.2.5 relate to whether the existing approaches focus on providing parallelism abstractions to application programmers. Although nowadays we have frameworks providing high-level programming abstractions for stream processing, 4 a limited number of approaches 13,16,43,53,54,68,74,90,95 mentioned abstractions as relevant for using/implementing self-adaptation. Considering that it tends to be very complex and time-consuming for application programmers to achieve self-adaptation in their domain-specific applications, we believe that the tools/frameworks should come with ready-to-use abstractions.…”

Section: Self-adaptive Parallelism In Stream Processingmentioning

confidence: 99%

See 3 more Smart Citations

Self‐adaptation on parallel stream processing: A systematic review

Vogel

Griebler

Danelutto

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

A recurrent challenge in real-world applications is autonomous management of the executions at run-time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long-running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self-adaptation can reduce costs and human efforts by providing a higher-level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self-adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self-adaptive approaches.Finally, applying the taxonomy made it possible to characterize the state-of-the-art, identify trends, and discuss open research challenges and future opportunities.

show abstract

“…The solution was validated emphasizing improvements in latency and resources consumption. This work was extended in Reference 74 for improving the performance and stability.…”

Section: Results Analysis and Discussionmentioning

confidence: 99%

Section: Results Analysis and Discussionmentioning

confidence: 99%

“…Consequently, reactive approaches may eventually violate QoS, but they tend to respond better to fluctuating workloads. References 37,47,63,65,66,74,75,82,98 claimed to be proactive.…”

Section: Adaptation Decisionsmentioning

confidence: 99%

Section: Self-adaptive Parallelism In Stream Processingmentioning

confidence: 99%

See 2 more Smart Citations

Self‐adaptation on parallel stream processing: A systematic review

Vogel

Griebler

Danelutto

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Kombi et al introduced [37] a holistic approach (DABS-Storm) that adapted the task requirements by dynamically controlling the resource usage as a latency aware load balancing strategy in stream processing systems.…”

Section: Related Workmentioning

confidence: 99%

Job scheduler for streaming applications in heterogeneous distributed processing systems

Al-Sinayyid

Zhu

2020

J Supercomput

View full text Add to dashboard Cite

In this study, we investigated the problem of scheduling streaming applications on a heterogeneous cluster environment and, based on our previous work, developed the maximum throughput scheduler algorithm (MT-Scheduler) for streaming applications. The proposed algorithm uses a dynamic programming technique to efficiently map the application topology onto the heterogeneous distributed system based on computing and data transfer requirements, while also taking into account the capacity of the underlying cluster resources. The proposed approach maximizes the system throughput by identifying and minimizing the time incurred at the computing/transfer bottleneck. The MT-Scheduler supports scheduling applications structured as a directed acyclic graph (DAG). We conducted experiments using three Storm microbenchmark topologies in both simulation and real Apache Storm environments. In terms of the performance evaluation, we compared the proposed MT-Scheduler with the simulated round robin and the default Storm scheduler algorithms. The results indicated that the MT-Scheduler outperforms the default round robin approach in terms of both the average system latency and throughput.

show abstract