Trading off space for passes in graph streaming problems

Abstract

Data stream processing has recently received increasing attention as a computational paradigm for dealing with massive data sets. Surprisingly, no algorithm with both sublinear space and passes is known for natural graph problems in classical read-only streaming. Motivated by technological factors of modern storage systems, some authors have recently started to investigate the computational power of less restrictive models where writing streams is allowed. In this paper, we show that the use of intermediate temporary streams is powerful enough to provide effective spacepasses tradeoffs for natural graph problems. In particular, for any space restriction of s bits, we show that single-source shortest paths in directed graphs with small positive integer edge weights can be solved in O((n log 3/2 n)/ √ s) passes. The result can be generalized to deal with multiple sources within the same bounds. This is the first known streaming algorithm for shortest paths in directed graphs. For undirected connectivity, we devise an O((n log n)/s) passes algorithm. Both problems require Ω(n/s) passes under the restrictions we consider. We also show that the model where intermediate temporary streams are allowed can be strictly more powerful than classical streaming for some problems, while maintaining all of its hardness for others.

Practical verified computation with streaming interactive proofs

“…The problem of determining if a graph is connected was considered in the standard stream model [48,64] and the multi-pass W-stream model [46]. In both models, it can be shown that any constant-pass algorithm without annotations needs Ω(n) bits of space.…”

An External-Memory Algorithm for String Graph Construction

“…Proof. First, we will consider a single generic iteration of the while loop at lines [8][9][10][11][12][13][14][15][16]. By Lemma 21 the intervals in Z are nested, therefore the intervals popped in a single iteration satisfy the lemma.…”

“…There are three main algorithmic solutions to cope with that amount of data: (1) data streaming, where only one pass is made over the data and the working memory is small compared to the input data [15,18], (2) parallel algorithms, where input data is split among several processors [30], and (3) external memory algorithms [2,32] where only part of the data is kept in main memory and most of the data are on disk.…”

Compact graph representations and parallel connectivity algorithms for massive dynamic network analysis

“…The semi-streaming model is a relaxation to the classical streaming model, that allows O(n polylog n) space and multiple passes over data. This is a simpler model for solving graph problems and several recent successes have been reported [11]. However, this work is far from complete; we require faster exact and approximate algorithms to analyze peta-and exascale data sets and to experimentally evaluate the proposed semistreaming algorithms on current architectures.…”