Minimizing Self-adaptation Overhead in Parallel Stream Processing for Multi-cores

Vogel, Adriano; Griebler, Dalvan; Danelutto, Marco; Fernandes, Luiz Gustavo

doi:10.1007/978-3-030-48340-1_3

Cited by 4 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work of Vogel et al 14 addressed another relevant facet of self‐adaptiveness: converge faster to a suitable configuration and minimize instability for reducing the decision‐making overhead. The proposed solution improved the decision‐making step and the evaluation with different applications and workloads show performance gains as well as lower overhead.…”

Section: Results Analysis and Discussionmentioning

confidence: 99%

“…Regarding RQ9 that related to the overhead category described in Section 4.3.3, the section "Evaluation-metrics" of Figure 4 provides results concerning the overhead measurement in the literature. The measurement of the overhead requires a comprehensive validation scenario, only the references 14,37,39,42,47,58,68,82,101 considered the overhead that can be caused by performing adaptation actions at run-time. Relevant aspects could be the resource utilization and performance of the application.…”

Section: Overhead Measurementmentioning

confidence: 99%

“…The design and practical implementation of self‐adaptation have been reported to be difficult 4,13 . Considering that self‐adaptation can affect execution's safety and cause overheads, 14 the evaluation and validation of the proposed self‐adaptive solutions for stream processing are of paramount importance. Thus, from the parallel stream processing standpoint, we argue that there is a demand to understand better what is being proposed and how the existing solutions are validated.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Self‐adaptation on parallel stream processing: A systematic review

Vogel

Griebler

Danelutto

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

A recurrent challenge in real-world applications is autonomous management of the executions at run-time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long-running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self-adaptation can reduce costs and human efforts by providing a higher-level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self-adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self-adaptive approaches.Finally, applying the taxonomy made it possible to characterize the state-of-the-art, identify trends, and discuss open research challenges and future opportunities.

show abstract

Section: Results Analysis and Discussionmentioning

confidence: 99%

Section: Overhead Measurementmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self‐adaptation on parallel stream processing: A systematic review

Vogel

Griebler

Danelutto

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…A system is expected to present short settling times by quickly responding to changes and reaching an optimal state. 30 When the load fluctuates, the response to an event should be rapid and maintain a service level objective. A strategy should also avoid overshooting by only using the necessary amount of resources.…”

Section: Design Considerationsmentioning

confidence: 99%

Providing high‐level self‐adaptive abstractions for stream parallelism on multicores

2021

Self Cite

View full text Add to dashboard Cite

Stream processing applications are common computing workloads that demand parallelism to increase their performance. As in the past, parallel programming remains a difficult task for application programmers. The complexity increases when application programmers must set nonintuitive parallelism parameters, that is, the degree of parallelism. The main problem is that state-of-the-art libraries use a static degree of parallelism and are not sufficiently abstracted for developing stream processing applications. In this article, we propose a self-adaptive regulation of the degree of parallelism to provide higher-level abstractions. Flexibility is provided to programmers with two new self-adaptive strategies, one is for performance experts, and the other abstracts the need to set a performance goal. We evaluated our solution using compiler transformation rules to generate parallel code with the SPar domain-specific language. The experimental results with real-world applications highlighted higher abstraction levels without significant performance degradation in comparison to static executions. The strategy for performance experts achieved slightly higher performance than the one that works without user-defined performance goals.

show abstract

“…The continuous data arrival requires stream processing applications to run for long or even infinite periods. These long executions are subject to occasional fluctuations in the environment (e.g., resource availability) and at the application level (input rate, workload trend) [2], [3]. Responsiveness at run-time is a potential way of coping with such a scenario.…”

Section: Introductionmentioning

confidence: 99%

Towards On-the-fly Self-Adaptation of Stream Parallel Patterns

Vogel¹,

Mencagli²,

Griebler³

et al. 2021

2021 29th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

Self Cite

View full text Add to dashboard Cite

Stream processing applications compute streams of data and provide insightful results in a timely manner, where parallel computing is necessary for accelerating the application executions. Considering that these applications are becoming increasingly dynamic and long-running, a potential solution is to apply dynamic runtime changes. However, it is challenging for humans to continuously monitor and manually self-optimize the executions. In this paper, we propose self-adaptiveness of the parallel patterns used, enabling flexible on-the-fly adaptations. The proposed solution is evaluated with an existing programming framework and running experiments with a synthetic and a realworld application. The results show that the proposed solution is able to dynamically self-adapt to the most suitable parallel pattern configuration and achieve performance competitive with the best static cases. The feasibility of the proposed solution encourages future optimizations and other applicabilities.1 A time period where a given application does not produce output 89

show abstract

Minimizing Self-adaptation Overhead in Parallel Stream Processing for Multi-cores

Cited by 4 publications

References 10 publications

Self‐adaptation on parallel stream processing: A systematic review

Self‐adaptation on parallel stream processing: A systematic review

Providing high‐level self‐adaptive abstractions for stream parallelism on multicores

Towards On-the-fly Self-Adaptation of Stream Parallel Patterns

Contact Info

Product

Resources

About