Gloss

Rajadurai, Sumanaruban; Bosboom, Jeffrey; Wong, Weng-Fai; Amarasinghe, Saman

doi:10.1145/3296957.3173170

Cited by 6 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ChronoStream [28] concurrently executes two or more computation slices and can migrate an arbitrary set of keys between instances of a single dataflow operator. Gloss [24] follows a similar approach and gathers operator state during a migration in a centralized controller using an asynchronous protocol.…”

Section: Dataflow Replicationmentioning

confidence: 99%

See 1 more Smart Citation

Megaphone

Hoffmann¹,

Lattuada

McSherry³

2019

Proc. VLDB Endow.

View full text Add to dashboard Cite

We design and implement Megaphone, a data migration mechanism for stateful distributed dataflow engines with latency objectives. When compared to existing migration mechanisms, Megaphone has the following differentiating characteristics: (i) migrations can be subdivided to a configurable granularity to avoid latency spikes, and (ii) migrations can be prepared ahead of time to avoid runtime coordination. Megaphone is implemented as a library on an unmodified timely dataflow implementation, and provides an operator interface compatible with its existing APIs. We evaluate Megaphone on established benchmarks with varying amounts of state and observe that compared to naïve approaches Megaphone reduces service latencies during reconfiguration by orders of magnitude without significantly increasing steady-state overhead.

show abstract

Section: Dataflow Replicationmentioning

confidence: 99%

“…parts of the dataflow (as in Flink [10], Dhalion [16], and SEEP [15]) or launch new dataflows alongside the old configuration (as for example in ChronoStream [28] and Gloss [24]). In both cases state moves "all-at-once," with either high latency or resource usage during the migration.…”

Section: Introductionmentioning

confidence: 99%

Megaphone

Hoffmann¹,

Lattuada

McSherry³

2019

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

Self‐adaptation on parallel stream processing: A systematic review

Vogel

Griebler

Danelutto

et al. 2021

Concurrency and Computation

View full text Add to dashboard Cite

A recurrent challenge in real-world applications is autonomous management of the executions at run-time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long-running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self-adaptation can reduce costs and human efforts by providing a higher-level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self-adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self-adaptive approaches.Finally, applying the taxonomy made it possible to characterize the state-of-the-art, identify trends, and discuss open research challenges and future opportunities.

show abstract

Providing high‐level self‐adaptive abstractions for stream parallelism on multicores

2021

View full text Add to dashboard Cite

Stream processing applications are common computing workloads that demand parallelism to increase their performance. As in the past, parallel programming remains a difficult task for application programmers. The complexity increases when application programmers must set nonintuitive parallelism parameters, that is, the degree of parallelism. The main problem is that state-of-the-art libraries use a static degree of parallelism and are not sufficiently abstracted for developing stream processing applications. In this article, we propose a self-adaptive regulation of the degree of parallelism to provide higher-level abstractions. Flexibility is provided to programmers with two new self-adaptive strategies, one is for performance experts, and the other abstracts the need to set a performance goal. We evaluated our solution using compiler transformation rules to generate parallel code with the SPar domain-specific language. The experimental results with real-world applications highlighted higher abstraction levels without significant performance degradation in comparison to static executions. The strategy for performance experts achieved slightly higher performance than the one that works without user-defined performance goals.

show abstract

Gloss

Cited by 6 publications

References 33 publications

Megaphone

Megaphone

Self‐adaptation on parallel stream processing: A systematic review

Providing high‐level self‐adaptive abstractions for stream parallelism on multicores

Contact Info

Product

Resources

About