STREAM: The Stanford Data Stream Management System

Arasu, Arvind; Babcock, Brian; Babu, Shivnath; Cieslewicz, John; Datar, Mayur; Ito, Keith; Motwani, Rajeev; Srivastava, Uma; Widom, Jennifer

doi:10.1007/978-3-540-28608-0_16

Cited by 209 publications

(162 citation statements)

References 8 publications

Supporting

Mentioning

160

Contrasting

Unclassified

Order By: Relevance

“…In the academic arena, there are a number of systems that support different stream abstractions such as Stream [47], [48], StreamMIT [49], Aurora [50], Flextream [51], River [52], Cayuga [53] and Naiad [54]. None of those academic approaches seems to be focused on meeting deadlines, although some of them could be easily extended with the infrastructure proposed in our article to support end-toend deadlines.…”

Section: Distributed Stream Processing Patternsmentioning

confidence: 99%

See 1 more Smart Citation

Patterns for Distributed Real-Time Stream Processing

Val

Fernandez-Garcia

Sánchez-Fernandez

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Abstract-In recent yeais, big data systems have become an active area of research and development. Stream processing is one of the potential application scenarios of big data systems where the goal is to process aconlinoous, high velochyflow of information items. High frequencytradirg (HFT) in stock markets ortrendirgtopicdetection in Twitter are som e examples of stream processing applications. In some cases (like, for instance, in HFT), these applications have end-t�nd qualhy-of-service reqlirements and may benefh from the usage� real-time techniques. Taking this into account, the present articl e analyzes, from the point� view of real-time systems, a set of patterns that can be used when implementing a stream processing application. For each pattern, we discuss its advantages and dsadvanlages, as well as its impact in application performance, measured as response time, maximum ill)Ut frequency and changes in utilization demands due to the pattern.

show abstract

Section: Distributed Stream Processing Patternsmentioning

confidence: 99%

“…The work could also be applied to a number of previous distributed stream technologies: Stream [47], [48], StreaMIT [49], Aurora [50], Flextream [51] and the novel Spark [11] and Storm [10] initiatives. These technologies may profit from the patterns described in order to meet their application deadlines in an efficient way.…”

Section: Introductionmentioning

confidence: 99%

Patterns for Distributed Real-Time Stream Processing

Val

Fernandez-Garcia

Sánchez-Fernandez

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…More specifically, one of the important continuous query optimization approaches is sharing operators across similar query plans [14,[25][26][27], which allows every necessary operator to be executed only once and avoids processing redundancy. However, existing topological operators do not take into consideration this sharing execution concept.…”

Section: Related Workmentioning

confidence: 99%

“…All new data will traverse through the query plan. While some approaches have been proposed to optimize the overall structure of query plans [14,15], there was not much discussion of how to improve the efficiency of computational-intensive geospatial operators in a push-based system.…”

Section: Introductionmentioning

confidence: 99%

AHS Model: Efficient Topological Operators for a Sensor Web Publish/Subscribe System

Huang

Liang

2017

IJGI

View full text Add to dashboard Cite

Abstract:The Worldwide Sensor Web has been applied for monitoring the physical world with spatial and temporal scales that were impossible in the past. With the development of sensor technologies and interoperable open standards, sensor webs generate tremendous volumes of priceless data, enabling scientists to observe previously unobservable phenomena. With its powerful monitoring capability, the sensor web is able to capture time-critical events and provide up-to-date information to support decision-making. In order to harvest the full potential of the sensor web, efficiently processing sensor web data and providing timely notifications are necessary. Therefore, we aim to design a software component applying the publish/subscribe communication model for the sensor web. However, as sensor web data are geospatial in nature, existing topological operators are inefficient when processing a large number of geometries. This paper presents the Aggregated Hierarchical Spatial Model (AHS model) to efficiently determine topological relationships between sensor data and predefined query objects. By using a predefined hierarchical spatial framework to index geometries, the AHS model can match new sensor data with all subscriptions in a single process to improve the query performance. Based on our evaluation results, the query latency of the AHS model increases 2.5 times more slowly than that of PostGIS. As a result, we believe that the AHS model is able to more efficiently process topological operators in a sensor web publish/subscribe system.

show abstract

“…Stream [27] is a Data Stream Management System (DSMS) that could potentially be used to enhance Heimdall Policy Monitor processing capabilities. It provides a language named CQL (Continuous Query Language) [28] that allows the filtering of relevant records in the data stream, thus enabling a number of performance optimizations.…”

Section: Related Workmentioning

confidence: 99%

A Scalable History-Based Policy Engine

Gama

Ribeiro

Ferreira

Seventh IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY'06)

View full text Add to dashboard Cite

Abstract-The increasing complexity and heterogeneity in distributed systems is drawing system administrators into applying usage and access control policy engines. Higher-level policy languages allow policy administrators to demarcate themselves from implementation details, thus focusing on business rule definition. More specifically, history-based policies allow the specification of rules based on events that occurred in the past, such as separation-of-duty related rules (e.g. an employee cannot both issue a voucher and approve the payment).Several policy engines already support history-based semantics. However, they either provide limited expressiveness in policy rules or they neglect critical scalability issues. Individual policy definitions are disregarded in storage and lookup implementations, thus ignoring the potential for important performance optimizations. Furthermore, purging meta-policy semantics are not provided, inducing the growth of the past event repository until policy evaluation becomes unmanageable.We present an extension to the Heimdall 1 system, a historyenabled policy engine which allows the definition, enforcement and auditing of history-based policies. This extension targets the scalability of Heimdall in practical environments, introducing an evaluation optimizer and the concept of purging meta-policy tags. An evaluation built on selected usage patterns corroborates the effectiveness of our approach, denoting encouraging performance results.

show abstract

STREAM: The Stanford Data Stream Management System

Cited by 209 publications

References 8 publications

Patterns for Distributed Real-Time Stream Processing

Patterns for Distributed Real-Time Stream Processing

AHS Model: Efficient Topological Operators for a Sensor Web Publish/Subscribe System

A Scalable History-Based Policy Engine

Contact Info

Product

Resources

About