JetStream: Enabling high throughput live event streaming on multi-site clouds

Tudoran, Radu; Costan, Alexandru; Nano, Olivier; Santos, Ivo; Soncu, Hakan; Antoniu, Gabriel

doi:10.1016/j.future.2015.01.016

Cited by 18 publications

(11 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing work aims to use the Internet edges by trying to place certain stream processing elements on micro data centres (often called Cloudlets [103]) closer to where the data is generated [104], transferring events to the cloud in batches 3 Execution of scale out/in operations are user-specified, not triggered by the system. [105], or by exploiting mobile devices in the fog for stream processing [106]. Proposed architecture aims to place data analysis tasks at the edge of the Internet in order to reduce the amount of data transferred from sources to the cloud, improve the end-to-end latency, or offload certain analyses from the cloud [43].…”

Section: Distributed and Hybrid Architecturementioning

confidence: 99%

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Assuno

Veith

Buyya

2018

Journal of Network and Computer Applications

248

View full text Add to dashboard Cite

Under several emerging application scenarios, such as in smart cities, operational monitoring of large infrastructure, wearable assistance, and Internet of Things, continuous data streams must be processed under very short delays. Several solutions, including multiple software engines, have been developed for processing unbounded data streams in a scalable and efficient manner. More recently, architecture has been proposed to use edge computing for data stream processing. This paper surveys state of the art on stream processing engines and mechanisms for exploiting resource elasticity features of cloud computing in stream processing. Resource elasticity allows for an application or service to scale out/in according to fluctuating demands. Although such features have been extensively investigated for enterprise applications, stream processing poses challenges on achieving elastic systems that can make efficient resource management decisions based on current load. Elasticity becomes even more challenging in highly distributed environments comprising edge and cloud computing resources. This work examines some of these challenges and discusses solutions proposed in the literature to address them.

show abstract

Section: Distributed and Hybrid Architecturementioning

confidence: 99%

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Assuno

Veith

Buyya

2018

Journal of Network and Computer Applications

248

View full text Add to dashboard Cite

show abstract

“…JetStream [31] proposes a set of strategies for efficient transfers of events between cloud data-centers. JetStream can self-adapt to the conditions of streams by modeling and monitoring a set of context parameters.…”

Section: B Communication Optimizationmentioning

confidence: 99%

Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems

Souza

Matteussi

Anjos

et al. 2018

2018 International Conference on High Performance Computing &Amp; Simulation (HPCS)

View full text Add to dashboard Cite

Stream Processing Engines (SPEs) have to support high data ingestion to ensure the quality and efficiency for the end-user or a system administrator. The data flow processed by SPE fluctuates over time, and requires real-time or near realtime resource pool adjustments (network, memory, CPU and other). This scenario leads to the problem known as skewed data production caused by the non-uniform incoming flow at specific points on the environment, resulting in slow down of applications caused by network bottlenecks and inefficient load balance. This work proposes Aten as a solution to overcome unbalanced data flows processed by Big Data Stream applications in heterogeneous systems. Aten manages data aggregation and data streams within message queues, assuming different algorithms as strategies to partition data flow over all the available computational resources. The paper presents preliminary results indicating that is possible to maximize the throughput and also provide low latency levels for SPEs.

show abstract

“…A complementary effort towards low latency also gained significant traction: the idea of optimizing batch processing to handle frequent mini-batches. This approach was popularized by Spark via DStreams [17], then followed by Apache Storm Trident and data management tools like JetStream [14].…”

Section: Stream Processingmentioning

confidence: 99%

Data Multiverse: The Uncertainty Challenge of Future Big Data Analytics

Tudoran

Nicolae

Brasche

2017

Semantic Keyword-Based Search on Structured Data Sources

Self Cite

View full text Add to dashboard Cite

Abstract. With the explosion of data sizes, extracting valuable insight out of big data becomes increasingly difficult. New challenges begin to emerge that complement traditional, long-standing challenges related to building scalable infrastructure and runtime systems that can deliver the desired level of performance and resource efficiency. This vision paper focuses on one such challenge, which we refer to as the analytics uncertainty: with so much

show abstract

JetStream: Enabling high throughput live event streaming on multi-site clouds

Cited by 18 publications

References 43 publications

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Aten: A Dispatcher for Big Data Applications in Heterogeneous Systems

Data Multiverse: The Uncertainty Challenge of Future Big Data Analytics

Contact Info

Product

Resources

About