Saber

Koliousis, Alexandros; Weidlich, Matthias; Fernandez, Raul Castro; Wolf, Alexander L.; Costa, Paolo; Pietzuch, Peter

doi:10.1145/2882903.2882906

Cited by 97 publications

(17 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Single Machine x Fernandez [45] GP Imp External Cloud x x Lohrmann [91] GP Imp Cluster, Cloud x x Zygouras [158] CEP Dec Cloud x Schneider [125] GP Dec Cloud x x Rive i [118] GP Imp Cluster x Mayer [96,99] CEP Imp Cloud x Wu [147] GP Imp (External) Cluster x Nasir [108,109] GP Imp Cluster x x Saleh [120] CEP Dec Cluster x x Koliousis [78] GP Dec GPU x x x Zacheilas [153] CEP Dec Cloud x Nakamura [107] GP Imp Fog x Gedik [53] GP Spark Streaming and Structured Streaming by Spark [154]: As Spark applications originally processed batches, the Spark Streaming extensions process streamed data in micro batches. Structured Streaming interprets data streams as an unbounded table where each new data item extends the table.…”

Section: Parallelizationmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Röger

Mayer

2019

ACM Comput. Surv.

View full text Add to dashboard Cite

Stream Processing (SP) has evolved as the leading paradigm to process and gain value from the high volume of streaming data produced e.g. in the domain of the Internet of ings. An SP system is a middleware that deploys a network of operators between data sources, such as sensors, and the consuming applications. SP systems typically face intense and highly dynamic data streams. Parallelization and elasticity enables SP systems to process these streams with continuously high quality of service. e current research landscape provides a broad spectrum of methods for parallelization and elasticity in SP. Each method makes speci c assumptions and focuses on particular aspects of the problem. However, the literature lacks a comprehensive overview and categorization of the state of the art in SP parallelization and elasticity, which is necessary to consolidate the state of the research and to plan future research directions on this basis. erefore, in this survey, we study the literature and develop a classi cation of current methods for both parallelization and elasticity in SP systems. or even di erent processing nodes in a shared-nothing cloud-based infrastructure. Frequent state synchronization must not hamper parallel processing, while the processing results have to remain consistent. Research proposes di erent approaches for parallel, stateless and stateful SP. ey di er in assumptions about the operator functions and state externalization mechanisms an SP system supports.is led to the development of a broad range of parallelization approaches tackling di erent problem cases.Second, how to continuously adapt the level of parallelization when the conditions of the SP operators, e.g. the workload or resources available, change at runtime. On the one hand, an SP system always needs enough resources to process the input data streams with a satisfying quality of service (QoS), e.g. latency or throughput. On the other hand, continuous provisioning of computing resources for peak workloads wastes resources at o -peak hours. us, an elastic SP system scales its resources according to the current need. Cloud computing provides on-demand resources to realize such elasticity [9]. e pay-as-you-go business model of cloud computing allows to cut costs by dynamically adapting the resource reservations to the needs of the SP system. It is challenging to strive the right balance between resource over-provisioning-which is costly, but is robust to workload uctuations-and on-demand scaling-which is cheap, but is vulnerable to sudden workload peaks. To this end, academia and industry developed elasticity methods. Again, they di er in their optimization objectives and assumptions about the operator parallelization model employed, the target system architecture, state management as well as timing and methodology.While there are many works that propose methods and solutions for speci c parallelization and elasticity problems in SP systems, there is a severe lack of overview, comparison, and classi cation of these methods. When we investigate...

show abstract

Section: Parallelizationmentioning

confidence: 99%

“…Koliousis et al propose SABER [78], an SP engine that manages query processing on heterogeneous hardware with CPU and GPU cores. A spli er rst splits the incoming event streams into batches of a xed size and assigns them to a processing unit, i.e., a CPU core or a GPU processor.…”

Section: Parallelization For General Streammentioning

confidence: 99%

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Röger

Mayer

2019

ACM Comput. Surv.

View full text Add to dashboard Cite

show abstract

“…However, in adjacent areas several investigations are available: in stream processing, e.g. [1,78], in big data processing, e.g. [27,80,129], and in SPARQL query evaluation, e.g.…”

Section: Parallelizing and Distributing The Stream Reasonersmentioning

confidence: 99%

Stream reasoning: A survey and outlook

et al. 2017

View full text Add to dashboard Cite

Stream reasoning studies the application of inference techniques to data characterised by being highly dynamic. It can find application in several settings, from Smart Cities to Industry 4.0, from Internet of Things to Social Media analytics. This year stream reasoning turns ten, and in this article we analyse its growth. In the first part, we trace the main results obtained so far, by presenting the most prominent studies. We start by an overview of the most relevant studies developed in the context of semantic web, and then we extend the analysis to include contributions from adjacent areas, such as database and artificial intelligence. Looking at the past is useful to prepare for the future: in the second part, we present a set of open challenges and issues that stream reasoning will face in the next future.

show abstract

“…Pane-based splitting has been proposed in stream processing systems [6,22]. For instance, when the max or median value of a window of 1 minute shall be computed, that window is split into 6 fragments of 10 seconds, the fragments' max or median values are computed in parallel, and the global window's value is aggregated from the fragments' results.…”

Section: Related Workmentioning

confidence: 99%

Minimizing Communication Overhead in Window-Based Parallel Complex Event Processing

Mayer

Tariq

Rothermel

2017

Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems

View full text Add to dashboard Cite

Distributed Complex Event Processing (DCEP) is a paradigm to infer the occurrence of complex situations in the surrounding world from basic events like sensor readings. In doing so, DCEP operators detect event patterns on their incoming event streams. To yield high operator throughput, data parallelization frameworks divide the incoming event streams of an operator into overlapping windows that are processed in parallel by a number of operator instances. In doing so, the basic assumption is that the di erent windows can be processed independently from each other. However, consumption policies enforce that events can only be part of one pattern instance; then, they are consumed, i.e., removed from further pattern detection. That implies that the constituent events of a pattern instance detected in one window are excluded from all other windows as well, which breaks the data parallelism between di erent windows. In this paper, we tackle this problem by means of speculation: Based on the likelihood of an event's consumption in a window, subsequent windows may speculatively suppress that event. We propose the SPECTRE framework for speculative processing of multiple dependent windows in parallel. Our evaluations show an up to linear scalability of SPECTRE with the number of CPU cores.

show abstract

Saber

Cited by 97 publications

References 44 publications

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

A Comprehensive Survey on Parallelization and Elasticity in Stream Processing

Stream reasoning: A survey and outlook

Minimizing Communication Overhead in Window-Based Parallel Complex Event Processing

Contact Info

Product

Resources

About