Interfaces for Stream Processing Systems

Alur, Rajeev; Mamouras, Konstantinos; Stanford, Caleb; Tannen, Val

doi:10.1007/978-3-319-95246-8_3

Cited by 5 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Linear sequences and multisets should certainly be included, but it would be desirable to generalize the notion of streams as much as possible. Recent works explore the idea of generalizing streams to encompass a large class of partial orders [13,85], but we will see later that this approach excludes many useful instances. Stream processing is the computational paradigm where the input is not presented in full at the beginning of the computation, but instead it is given in an incremental fashion or piece by piece.…”

Section: Monoids As Types For Streamsmentioning

confidence: 99%

“…These streams can be organized elegantly using the structure of monoids. The sequences of Example 1, the multisets of Example 2, and the finite time-varying multisets of Example 7 can be described equivalently in terms of the partial orders of [13,85], which have also been suggested as an approach to unify notions of streams. Using partial orders it is also possible to model the timed finite sequences of Example 6, but only with a non-succinct encoding: every time punctuation t ∈ N is encoded with a sequence 11 .…”

Section: Example 5 (Bounded-domain Continuous-time Signals)mentioning

confidence: 99%

“…The idea of using types to classify streams has been recently explored in [85] (see also [13]), but only for a restricted class of types that correspond to partial orders. No general abstract model of computation is presented in [85], and many of the examples in this paper cannot be adequately accomodated.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Semantic Foundations for Deterministic Dataflow and Stream Processing

Mamouras

2020

Programming Languages and Systems

Self Cite

View full text Add to dashboard Cite

We propose a denotational semantic framework for deterministic dataflow and stream processing that encompasses a variety of existing streaming models. Our proposal is based on the idea that data streams, stream transformations, and stream-processing programs should be classified using types. The type of a data stream is captured formally by a monoid, an algebraic structure with a distinguished binary operation and a unit. The elements of a monoid model the finite fragments of a stream, the binary operation represents the concatenation of stream fragments, and the unit is the empty fragment. Stream transformations are modeled using monotone functions on streams, which we call stream transductions. These functions can be implemented using abstract machines with a potentially infinite state space, which we call stream transducers. This abstract typed framework of stream transductions and transducers can be used to (1) verify the correctness of streaming computations, that is, that an implementation adheres to the desired behavior, (2) prove the soundness of optimizing transformations, e.g. for parallelization and distribution, and (3) inform the design of programming models and query languages for stream processing. In particular, we show that several useful combinators can be supported by the full class of stream transductions and transducers: serial composition, parallel composition, and feedback composition.

show abstract

Section: Monoids As Types For Streamsmentioning

confidence: 99%

Section: Example 5 (Bounded-domain Continuous-time Signals)mentioning

confidence: 99%

See 1 more Smart Citation

Semantic Foundations for Deterministic Dataflow and Stream Processing

Mamouras

2020

Programming Languages and Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…We will introduce a semantic framework for distributed stream processing systems, where the input and output streams are viewed as partial orders [13]. Under this view, finite prefixes of streams are represented as data traces , and they are classified according to their ordering characteristics using types .…”

Section: Types For Data Streamsmentioning

confidence: 99%

“…Definition 3.5 essentially says that a data-string transduction f is consistent when it gives equivalent cumulative outputs for equivalent input sequences. It is shown in [13] that the set of data-trace transductions from X to Y is equal to the set of ( X , Y )-denotations of all ( X , Y )-consistent data-string transductions.…”

Section: Types For Data Streamsmentioning

confidence: 99%

Data-trace types for distributed stream processing systems

Mamouras

Stanford

Alur

et al. 2019

Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

Self Cite

View full text Add to dashboard Cite

Distributed architectures for efficient processing of streaming data are increasingly critical to modern information processing systems. The goal of this paper is to develop type-based programming abstractions that facilitate correct and efficient deployment of a logical specification of the desired computation on such architectures. In the proposed model, each communication link has an associated type specifying tagged data items along with a dependency relation over tags that captures the logical partial ordering constraints over data items. The semantics of a (distributed) stream processing system is then a function from input data traces to output data traces, where a data trace is an equivalence class of sequences of data items induced by the dependency relation. This data-trace transduction model generalizes both acyclic synchronous data-flow and relational query processors, and can specify computations over data streams with a rich variety of partial ordering and synchronization characteristics. We then describe a set of programming templates for data-trace transductions: abstractions corresponding to common stream processing tasks. Our system automatically maps these high-level programs to a given topology on the distributed implementation platform Apache Storm while preserving the semantics. Our experimental evaluation shows that (1) while automatic parallelization deployed by existing systems may not preserve semantics, particularly when the computation is sensitive to the ordering of data items, our programming abstractions allow a natural specification of the query that contains a mix of ordering constraints while guaranteeing correct deployment, and (2) the throughput of the automatically compiled distributed code is comparable to that of hand-crafted distributed implementations.

show abstract

Algebraic aspects of big data in livestream in internet of mobile things

Van Pham,

Phan

2024

Mobile Netw Appl

View full text Add to dashboard Cite

Interfaces for Stream Processing Systems

Cited by 5 publications

References 15 publications

Semantic Foundations for Deterministic Dataflow and Stream Processing

Semantic Foundations for Deterministic Dataflow and Stream Processing

Data-trace types for distributed stream processing systems

Algebraic aspects of big data in livestream in internet of mobile things

Contact Info

Product

Resources

About