Equivalence classes and conditional hardness in massively parallel computations

Abstract

The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale data processing frameworks, and has been receiving increasingly more attention over the past few years, especially in the context of classical graph problems. So far, the only way to argue lower bounds for this model is to condition on conjectures about the hardness of some specific problems, such as graph connectivity on promise graphs that are either one cycle or two cycles, usually called the one cycle versus two cycles problem. This is unlike the traditional arguments based on conjectures about complexity classes (e.g., $$\textsf {P}\ne \textsf {NP}$$ P ≠ NP ), which are often more robust in the sense that refuting them would lead to groundbreaking algorithms for a whole bunch of problems. In this paper we present connections between problems and classes of problems that allow the latter type of arguments. These connections concern the class of problems solvable in a sublogarithmic amount of rounds in the MPC model, denoted by $$\textsf {MPC}(o(\log N))$$ MPC ( o ( log N ) ) , and the standard space complexity classes $$\textsf {L}$$ L and $$\textsf {NL}$$ NL , and suggest conjectures that are robust in the sense that refuting them would lead to many surprisingly fast new algorithms in the MPC model. We also obtain new conditional lower bounds, and prove new reductions and equivalences between problems in the MPC model. Specifically, our main results are as follows. Lower bounds conditioned on the one cycle versus two cycles conjecture can be instead argued under the $$\textsf {L}\nsubseteq \textsf {MPC}(o(\log N))$$ L ⊈ MPC ( o ( log N ) ) conjecture: these two assumptions are equivalent, and refuting either of them would lead to $$o(\log N)$$ o ( log N ) -round MPC algorithms for a large number of challenging problems, including list ranking, minimum cut, and planarity testing. In fact, we show that these problems and many others require asymptotically the same number of rounds as the seemingly much easier problem of distinguishing between a graph being one cycle or two cycles. Many lower bounds previously argued under the one cycle versus two cycles conjecture can be argued under an even more robust (thus harder to refute) conjecture, namely $$\textsf {NL}\nsubseteq \textsf {MPC}(o(\log N))$$ NL ⊈ MPC ( o ( log N ) ) . Refuting this conjecture would lead to $$o(\log N)$$ o ( log N ) -round MPC algorithms for an even larger set of problems, including all-pairs shortest paths, betweenness centrality, and all aforementioned ones. Lower bounds under this conjecture hold for problems such as perfect matching and network flow.

Massively Parallel Computation and Sublinear-Time Algorithms for Embedded Planar Graphs

“…This has been proven for some special classes of algorithms; for an overview of these results, see [28,Section 3.2]. This conjecture is known to be equivalent to many common combinatorial problems [28]. Specifically, any of the problems that we consider in Section 5.2 need Ω(log n) rounds with strongly sublinear space if this conjecture holds.…”

“…The 1-vs-2-cycles conjecture states that for any > 0, one needs Ω(log n) rounds to tell apart one cycle of length n and two cycles of length n/2 when using n 1−Ω(1) space per machine. This has been proven for some special classes of algorithms; for an overview of these results, see [28,Section 3.2]. This conjecture is known to be equivalent to many common combinatorial problems [28].…”

Deterministic massively parallel connectivity

“…One of the notorious problems for low space MPC is the problem of distinguishing whether an input graph is an 𝑛-vertex cycle or two 𝑛 2 -vertex cycles. This fundamental MPC problem has been studied extensively in the literature and has played a central role in the development of conditional lower bounds for MPC graph algorithms (see, e.g., [13,45,52,56]). This problem is trivial for an MPC with local space Ω(𝑛), but we do not know its complexity for low space MPCs.…”

On the Hardness of Massively Parallel Computation

“…Under the same conjecture, Yaroslavtsev et al [65] showed that single-linkage clustering cannot be approximated by constant factor in o(log n) rounds. Recently, Nanongkai and Scquizzato showed that this conjecture is equivalent to the conjecture that logspace complete problem can not be solved in o(log n) rounds, and consequently, a large class of graph problems, such as single source shortest path, minimum cut, and planarity testing, require asymptotically the same number of rounds under these assumptions [57].…”