Near-Optimal Massively Parallel Graph Connectivity

Behnezhad, Soheil; Dhulipala, Laxman; Esfandiari, Hossein; Łącki, Jakub; Mirrokni, Vahab

doi:10.1109/focs.2019.00095

Cited by 30 publications

(87 citation statements)

References 53 publications

(67 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We note that for D = 2 o(log n/log log n) , the total number of random bits used by our algorithm is (log n) O (R) = o(n δ ) for any arbitrarily small constant δ > 0. Thus our algorithm achieves a super-polynomial saving in randomness complexity as compared to the algorithms of [1,5].…”

Section: This Work: Randomness-efficient Mpc Algorithm For Connectivitymentioning

confidence: 94%

“…We are primarily interested in designing randomness-efficient algorithms in the MPC model. For Connectivity, the aforementioned randomized MPC algorithms of [1,5] use Ω(n) random bits. Our main result is the following: Theorem 1 (Randomness-efficient MPC algorithm for Connectivity).…”

Section: This Work: Randomness-efficient Mpc Algorithm For Connectivitymentioning

confidence: 99%

“…If O(m 1+Ω(1) ) total space is allowed, their algorithm runs in O(log D) rounds. This was followed by the work of Behnezhad et al [5], who improved the round complexity to O(log D + log log m/n n) while using O(m) total space.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Brief Announcement

Charikar

Tan

2021

Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

We give a randomness-efficient Massively Parallel Computation (MPC) algorithm for deciding whether an undirected graph is connected. For Connectivity on n-vertex, m-edge graphs whose components have diameter at most D = 2 o(log n/log log n) , our algorithm runs in R = O(log D + log log m/n n) rounds and uses a total of (log n) O (R) random bits, O(m) machines, and n 1−Ω(1) space per machine with good probability. 1 Our algorithm achieves a superpolynomial saving in randomness complexity as compared to the breakthrough algorithm of Andoni et al. (FOCS '18) and the subsequent improvement by Behnezhad et al. (FOCS '19). Our algorithm has the same round complexity as that of Behnezhad et al., but uses more total space.Our Connectivity algorithm is an instantiation of a general method we develop for converting randomized algorithms in the PRAM model to highly randomness-efficient MPC algorithms. We show that for k = o(log n/log log n) and p = n O (1) , any time-k pprocessor randomized PRAM algorithm computing a function on n input bits can be converted to an equivalent strongly sublinear MPC algorithm with O(k) rounds and only a total of (log n) O (k ) random bits. Our Connectivity algorithm follows from applying this method to the recent CRCW PRAM algorithm of Liu, Tarjan, and Zhong (SPAA '20).Our approach is based on the design of a pseudorandom generator for PRAM algorithms. The analysis of our generator is built on classic and influential results in circuit complexity (Håstad '86; Nisan and Wigderson '88), which we generalize from the setting of small-depth circuits to the more powerful setting of PRAM algorithms. The parameters that we achieve are optimal given the current state of the art in complexity theory, in the sense that further improvements will imply P NC 1 . CCS CONCEPTS• Mathematics of computing → Graph algorithms; • Theory of computation → Massively parallel algorithms; Pseudorandomness and derandomization.1 With good probability means with probability at least 1 − 1/poly((m log n)/n), which is the same as in Liu, Tarjan, and Zhong (SPAA '20).

show abstract

Section: This Work: Randomness-efficient Mpc Algorithm For Connectivitymentioning

confidence: 94%

Section: This Work: Randomness-efficient Mpc Algorithm For Connectivitymentioning

confidence: 99%

See 1 more Smart Citation

Brief Announcement

Charikar

Tan

2021

Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

show abstract

“…When m = o(n log 2 n) we use the algorithm of [11] to shrink the number of vertices by a factor of Ω(log 2 n) in O(log log T /n n) rounds w.h.p. This procedure is implementable in the MPC model and hence implementable in the AMPC model as well.…”

Section: Returnmentioning

confidence: 99%

“…[11]). There exists an O(log log n) round MPC algorithm using O(n ) space per machine and O(m) total space that with high probability, converts any graph G(V, E) with n vertices and m edges to a graph G (V , E ) and outputs a function f : V → V such that:1.…”

mentioning

confidence: 99%

Massively Parallel Computation via Remote Memory Access

Behnezhad

Dhulipala

Esfandiari

et al. 2019

The 31st ACM Symposium on Parallelism in Algorithms and Architectures

Self Cite

View full text Add to dashboard Cite

We introduce the Adaptive Massively Parallel Computation (AMPC) model, which is an extension of the Massively Parallel Computation (MPC) model. At a high level, the AMPC model strengthens the MPC model by storing all messages sent within a round in a distributed data store. In the following round, all machines are provided with random read access to the data store, subject to the same constraints on the total amount of communication as in the MPC model. Our model is inspired by the previous empirical studies of distributed graph algorithms [28,9] using MapReduce and a distributed hash table service [17].This extension allows us to give new graph algorithms with much lower round complexities compared to the best known solutions in the MPC model. In particular, in the AMPC model we show how to solve maximal independent set in O(1) rounds and connectivity/minimum spanning tree in O(log log m/n n) rounds both using O(n δ ) space per machine for constant δ < 1. In the same memory regime for MPC, the best known algorithms for these problems require poly log n rounds. Our results imply that the 2-Cycle conjecture, which is widely believed to hold in the MPC model, does not hold in the AMPC model.

show abstract

Equivalence classes and conditional hardness in massively parallel computations

Nanongkai

Scquizzato

2022

Distrib. Comput.

View full text Add to dashboard Cite

The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of classical graph problems. So far, the only way to argue lower bounds for this model is to condition on conjectures about the hardness of some specific problems, such as graph connectivity on promise graphs that are either one cycle or two cycles, usually called the one cycle versus two cycles problem. This is unlike the traditional arguments based on conjectures about complexity classes (e.g., $$\textsf {P}\ne \textsf {NP}$$ P ≠ NP ), which are often more robust in the sense that refuting them would lead to groundbreaking algorithms for a whole bunch of problems. In this paper we present connections between problems and classes of problems that allow the latter type of arguments. These connections concern the class of problems solvable in a sublogarithmic amount of rounds in the MPC model, denoted by $$\textsf {MPC}(o(\log N))$$ MPC ( o ( log N ) ) , and the standard space complexity classes $$\textsf {L}$$ L and $$\textsf {NL}$$ NL , and suggest conjectures that are robust in the sense that refuting them would lead to many surprisingly fast new algorithms in the MPC model. We also obtain new conditional lower bounds, and prove new reductions and equivalences between problems in the MPC model. Specifically, our main results are as follows. Lower bounds conditioned on the one cycle versus two cycles conjecture can be instead argued under the $$\textsf {L}\nsubseteq \textsf {MPC}(o(\log N))$$ L ⊈ MPC ( o ( log N ) ) conjecture: these two assumptions are equivalent, and refuting either of them would lead to $$o(\log N)$$ o ( log N ) -round MPC algorithms for a large number of challenging problems, including list ranking, minimum cut, and planarity testing. In fact, we show that these problems and many others require asymptotically the same number of rounds as the seemingly much easier problem of distinguishing between a graph being one cycle or two cycles. Many lower bounds previously argued under the one cycle versus two cycles conjecture can be argued under an even more robust (thus harder to refute) conjecture, namely $$\textsf {NL}\nsubseteq \textsf {MPC}(o(\log N))$$ NL ⊈ MPC ( o ( log N ) ) . Refuting this conjecture would lead to $$o(\log N)$$ o ( log N ) -round MPC algorithms for an even larger set of problems, including all-pairs shortest paths, betweenness centrality, and all aforementioned ones. Lower bounds under this conjecture hold for problems such as perfect matching and network flow.

show abstract

Near-Optimal Massively Parallel Graph Connectivity

Cited by 30 publications

References 53 publications

Brief Announcement

Brief Announcement

Massively Parallel Computation via Remote Memory Access

Equivalence classes and conditional hardness in massively parallel computations

Contact Info

Product

Resources

About