TimeStream

Qian, Zhuzhong; He, You; Su, Chunzhi; Wu, Zhuo-Jie; Zhu, Hongyu; Zhang, Taizhi; Zhou, Lidong; Yu, Yuan; Zhang, Zheng

doi:10.1145/2465351.2465353

Cited by 180 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other systems, such as TimeStream [70], use a DAG abstraction for structuring an application as a graph of operators that execute user-defined functions. Employing a graph abstraction is not exclusive to data stream processing.…”

Section: Other Solutionsmentioning

confidence: 99%

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Assuno

Veith

Buyya

2018

Journal of Network and Computer Applications

254

View full text Add to dashboard Cite

Under several emerging application scenarios, such as in smart cities, operational monitoring of large infrastructure, wearable assistance, and Internet of Things, continuous data streams must be processed under very short delays. Several solutions, including multiple software engines, have been developed for processing unbounded data streams in a scalable and efficient manner. More recently, architecture has been proposed to use edge computing for data stream processing. This paper surveys state of the art on stream processing engines and mechanisms for exploiting resource elasticity features of cloud computing in stream processing. Resource elasticity allows for an application or service to scale out/in according to fluctuating demands. Although such features have been extensively investigated for enterprise applications, stream processing poses challenges on achieving elastic systems that can make efficient resource management decisions based on current load. Elasticity becomes even more challenging in highly distributed environments comprising edge and cloud computing resources. This work examines some of these challenges and discusses solutions proposed in the literature to address them.

show abstract

Section: Other Solutionsmentioning

confidence: 99%

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Assuno

Veith

Buyya

2018

Journal of Network and Computer Applications

254

View full text Add to dashboard Cite

show abstract

“…These have evolved into a general purpose abstraction that provides strong composition tools for building large scale stream processing applications. Several such stream processing systems have been recently proposed such as S4 [4], IBM InfoSphere streams [17], D-Streams [5], Storm [24], and Time Stream [6]. While these systems are built for large scale stream processing applications and provide distributed deployment and runtime, most of these (with the exception of Time Stream) do not provide automatic, dynamic adaptations to changing data rates or performance variability of the underlying infrastructure, over the application's lifetime.…”

Section: Related Work a Continuous Dataflowsmentioning

confidence: 99%

“…However, unlike these systems, where the alternate to be executed is selected in advance, in our dynamic continuous dataflow abstraction such decisions are made at periodic intervals based on the changing execution characteristics. Time Stream [6] supports similar dynamic reconfiguration, called resilient substitution, which allows replacing a subgraph with another in response to the changing load. This requires dependency tracking and introduces inefficiencies.…”

Section: B Dynamic Adaptations and Scalingmentioning

confidence: 99%

“…Their elastic resource provisioning is well suited to scale resources at runtime in response to changing needs of continuous applications. Several stream processing frameworks [3], [4], [5] and adaptive provisioning algorithms [6], including our own earlier work [7], [8], utilize this online elasticity to balance resource cost against application performance in response to changing data rates.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PLAStiCC: Predictive Look-Ahead Scheduling for Continuous Dataflows on Clouds

Kumbhare

Simmhan

Prasanna

2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

Abstract-Scalable stream processing and continuous dataflow systems are gaining traction with the rise of big data due to the need for processing high velocity data in near real time. Unlike batch processing systems such as MapReduce and workflows, static scheduling strategies fall short for continuous dataflows due to the variations in the input data rates and the need for sustained throughput. The elastic resource provisioning of cloud infrastructure is valuable to meet the changing resource needs of such continuous applications. However, multi-tenant cloud resources introduce yet another dimension of performance variability that impacts the application's throughput. In this paper we propose PLAStiCC, an adaptive scheduling algorithm that balances resource cost and application throughput using a prediction-based lookahead approach. It not only addresses variations in the input data rates but also the underlying cloud infrastructure. In addition, we also propose several simpler static scheduling heuristics that operate in the absence of accurate performance prediction model. These static and adaptive heuristics are evaluated through extensive simulations using performance traces obtained from public and private IaaS clouds. Our results show an improvement of upto 20% in the overall profit as compared to the reactive adaptation algorithm.

show abstract

“…Popular open source systems in this category include Storm (https://storm.incubator.apache.org/) and S4 (http://incubator.apache.org/s4/), and Spark Streaming [28]. TimeStream [18] addresses efficient failure recovery in operatornetwork streaming systems. Resa [24] provides enhanced elasticity, and applies to a wider range of analytics such as clustering [29].…”

Section: Related Workmentioning

confidence: 99%

Design and implementation of a real-time interactive analytics system for large spatio-temporal data

et al. 2014

View full text Add to dashboard Cite

In real-time interactive data analytics, the user expects to receive the results of each query within a short time period such as seconds. This is especially challenging when the data is big (e.g., on the scale of petabytes), and the analytics system runs on top of cloud infrastructure (e.g., thousands of interconnected commodity servers). We have been building such a system, called OceanRT, for managing large spatio-temporal data such as call logs and mobile web browsing records collected by a telecommunication company. Although there already exist systems for querying big data in real time, OceanRT's performance stands out due to several novel designs and components that address key efficiency and scalability issues that were largely overlooked in existing systems. First, OceanRT makes extensive use of software RDMA one-sided operations, which reduce networking costs without requiring specialized hardware. Second, OceanRT exploits the parallel computing capabilities of each node in the cloud through a novel architecture consisting of Access-Query Engines (AQEs) connected with minimal overhead. Third, OceanRT contains a novel storage scheme that optimizes for queries with joins and multi-dimensional selections, which are common for large spatiotemporal data. Experiments using the TPC-DS benchmark show that OceanRT is usually more than an order of magnitude faster than the current state-of-the-art systems.

show abstract

TimeStream

Cited by 180 publications

References 19 publications

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

Distributed data stream processing and edge computing: A survey on resource elasticity and future directions

PLAStiCC: Predictive Look-Ahead Scheduling for Continuous Dataflows on Clouds

Design and implementation of a real-time interactive analytics system for large spatio-temporal data

Contact Info

Product

Resources

About