Trading off space for passes in graph streaming problems

Abstract

Data stream processing has recently received increasing attention as a computational paradigm for dealing with massive data sets. While major progress has been achieved for several fundamental data sketching and statistics problems, there are many problems that seem to be hard in a streaming setting, including most classical graph problems. Relevant examples are graph connectivity and shortest paths, for which linear lower bounds on the "space x passes" product are known. Some recent papers have shown that several graph problems can be solved with one or few passes, if the working memory is large enough to contain all the vertices of the graph. A natural question is whether we can reduce the space usage at the price of increasing the number of passes. Surprisingly, no algorithm with both sublinear space and passes is known for natural graph problems in classical streaming models. Motivated by technological factors of modern storage systems, some authors have recently started to investigate the computational power of less restrictive streaming models. In a FOCS'04 paper, Aggarwal et al. have shown that the use of intermediate temporary streams, combined with the ability to reorder them at each pass for free, yields enough power to solve in polylogarithmic space and passes a variety of problems, including graph connectivity. They leave however as an open question whether problems such as shortest paths can be solved efficiently in this more powerful model. In this paper, we show that the "streaming with sorting" model by Aggarwal et al. can yield interesting results even without using sorting at all: by just using intermediate temporary streams, we provide the first effective spacepasses tradeoffs for natural graph problems. In particular, for any space restriction of s bits, we show that single-source shortest paths in directed graphs with small positive integer edge weights can be solved in O((n log(3/2) n)/root 7s) passes. This is the first known streaming algorithm for shortest paths in directed graphs. For undirected connectivity, we devise an O((n log n)/s) passes algorithm. Both problems require Omega(n/s) passes under the restrictions we consider. We also show that the model where intermediate temporary streams are allowed can be strictly more powerful than classical streaming for some problems, while maintaining all of its hardness for others

New Bounds for the CLIQUE-GAP Problem Using Graph Decomposition Theory

“…Let X ⊂ E be the minimum edge set that any triangle t ∈ T (u) touches an edge in X . We claim |X | = Ω(n β ) by (12), (13) and by…”

Listing Dense Subgraphs in Small Memory

“…They use edge weights of edges in the stream and compare them with the sum of the weights of the edges in the current matching M . Demetrescu et al [8] show that the single-source shortest path problem in directed graphs can be solved in the w-stream model in O(n log 3/2 n)/ √ s) passes. For undirected connectivity they propose an O(n log n)/s passes algorithm.…”

Approximate Shortest Paths Guided by a Small Index

“…Interestingly, for many of these properties, such as connectedness, which is at the basis of short path computations, at least Ω(n) bits need to be stored if streams can only be read [10]. The W-Stream model [7], where intermediate streams are written in one pass to be read in the next pass, allows for a trade-off between memory and number of passes for single source shortest paths. Such an intermediate stream can be viewed as an excerpt of the graph.…”