Incremental, iterative data processing with timely dataflow

Murray, Derek G.; McSherry, Frank; Isard, Michael; Isaacs, Rebecca; Barham, Paul; Abadi, Martı́n

doi:10.1145/2983551

Cited by 33 publications

(19 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, conditional computation, where parts of a neural network are active on a per-example basis, has been proposed as a way to increase model capacity without a proportional increase in computation; recent research has demonstrated architectures with over 100 billion parameters [38]. Also, continuous training and inference may rely on streaming systems, perhaps using the dynamic dataflow architectures that we adapt, as suggested by work on timely dataflow and differential dataflow [28]. Further research on abstractions and implementation techniques for conditional and streaming computation seems worthwhile.…”

Section: Discussionmentioning

confidence: 99%

Dynamic control flow in large-scale machine learning

Yuan

Abadi

Barham

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

Self Cite

View full text Add to dashboard Cite

Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from the ability to make rapid control-flow decisions across a set of computing devices in a distributed system. For performance, scalability, and expressiveness, a machine learning system must support dynamic control flow in distributed and heterogeneous environments. This paper presents a programming model for distributed machine learning that supports dynamic control flow. We describe the design of the programming model, and its implementation in TensorFlow, a distributed machine learning system. Our approach extends the use of dataflow graphs to represent machine learning models, offering several distinctive features. First, the branches of conditionals and bodies of loops can be partitioned across many machines to run on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs. Second, programs written in our model support automatic differentiation and distributed gradient computations, which are necessary for training machine learning models * Work done primarily at Google Brain. that use control flow. Third, our choice of non-strict semantics enables multiple loop iterations to execute in parallel across machines, and to overlap compute and I/O operations.We have done our work in the context of TensorFlow, and it has been used extensively in research and production. We evaluate it using several real-world applications, and demonstrate its performance and scalability.

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamic control flow in large-scale machine learning

Yuan

Abadi

Barham

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…Dataflow-based computational models were proposed to perform complex analytics on high-volume data sets: the timely dataflow [69,70] model targets batch processing, while its extension, differential dataflow [65], targets incremental processing.…”

Section: Tool Descriptionmentioning

confidence: 99%

“…However, the notion of time during the calculation of the query allows this tools to support temporary state, in particular the iterate call in Listing 29. This temporary state allows to support also more complex cases such as computing connected components in Q2 using only built-in algorithms and data structures [70].…”

Section: Explicitness Of Incrementalizationmentioning

confidence: 99%

A cross-technology benchmark for incremental graph queries

Hinkel¹,

García-Domínguez

Schöne

et al. 2021

Softw Syst Model

View full text Add to dashboard Cite

To cope with the increased complexity of systems, models are used to capture what is considered the essence of a system. Such models are typically represented as a graph, which is queried to gain insight into the modelled system. Often, the results of these queries need to be adjusted according to updated requirements and are therefore a subject of maintenance activities. It is thus necessary to support writing model queries with adequate languages. However, in order to stay meaningful, the analysis results need to be refreshed as soon as the underlying models change. Therefore, a good execution speed is mandatory in order to cope with frequent model changes. In this paper, we propose a benchmark to assess model query technologies in the presence of model change sequences in the domain of social media. We present solutions to this benchmark in a variety of 11 different tools and compare them with respect to explicitness of incrementalization, asymptotic complexity and performance.

show abstract

“…Specifically, research in dynamic query processing has recently received a big boost with: (1) the introduction of Higher-Order IVM (HIVM) [14,19,13]; (2) the identification of lower bounds and worst-case optimal algorithms for processing updates [3,4,12]; (3) the practical formulations of worst-case-optimal IVM that implement and extend these algorithms [10,11,20]; and (4) the introduction of the notion of differential dataflow for computations that require recursive or iterative processing [17,16]. These approaches often rely on materializing a succinct representation of a query's output to maintain it more efficiently, and therefore present a fundamental breakthrough with traditional IVM techniques.…”

Section: Introductionmentioning

confidence: 99%

Incremental Techniques for Large-Scale Dynamic Query Processing

Elghandour

Kara

Olteanu

et al. 2018

Proceedings of the 27th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Many applications from various disciplines are now required to analyze fast evolving big data in real time. Various approaches for incremental processing of queries have been proposed over the years. Traditional approaches rely on updating the results of a query when updates are streamed rather than re-computing these queries, and therefore, higher execution performance is expected. However, they do not perform well for large databases that are updated at high frequencies. Therefore, new algorithms and approaches have been proposed in the literature to address these challenges by, for instance, reducing the complexity of processing updates. Moreover, many of these algorithms are now leveraging distributed streaming platforms such as Spark Streaming and Flink. In this tutorial, we briefly discuss legacy approaches for incremental query processing, and then give an overview of the new challenges introduced due to processing big data streams. We then discuss in detail the recently proposed algorithms that address some of these challenges. We emphasize the characteristics and algorithmic analysis of various proposed approaches and conclude by discussing future research directions.

show abstract

Incremental, iterative data processing with timely dataflow

Cited by 33 publications

References 20 publications

Dynamic control flow in large-scale machine learning

Dynamic control flow in large-scale machine learning

A cross-technology benchmark for incremental graph queries

Incremental Techniques for Large-Scale Dynamic Query Processing

Contact Info

Product

Resources

About