On the Hardness of Massively Parallel Computation

Abstract

We investigate whether there are inherent limits of parallelization in the (randomized) massively parallel computation (MPC) model by comparing it with the (sequential) RAM model. As our main result, we show the existence of hard functions that are essentially not parallelizable in the MPC model. Based on the widely-used random oracle methodology in cryptography with a cryptographic hash function h : {0, 1} n → {0, 1} n computable in time t h , we show that there exists a function that can be computed in time O(T • t h) and space S by a RAM algorithm, but any MPC algorithm with local memory size s < S/c for some c > 1 requires at least Ω(T) 1 rounds to compute the function, even in the average case, for a wide range of parameters n ≤ S ≤ T ≤ 2 n 1/4. Our result is almost optimal in the sense that by taking T to be much larger than t h , e.g., T to be sub-exponential in t h , to compute the function, the round complexity of any MPC algorithm with small local memory size is asymptotically the same (up to a polylogarithmic factor) as the time complexity of the RAM algorithm. Our result is obtained by adapting the so-called compression argument from the data structure lower bounds and cryptography literature to the context of massively parallel computation.

Equivalence classes and conditional hardness in massively parallel computations

“…Assuming the same conjecture, Behnezhad et al [19] show a parameterized lower bound of Ω(log D) for identifying connected components in graphs of diameter D. By observing that a couple of specific NC 1 reductions can be simulated in O(1) MPC rounds, Dhulipala et al [36] show that if a variant of graph connectivity on batch-dynamic graphs can be solved within a certain amount of rounds, so can all the problems in P. A conditional lower bound following a different kind of argument is given by Andoni et al [8], who show that an n o (1) -round MPC algorithm that answers O(n + m) pairs of reachability queries in directed graphs with n nodes and m edges can be simulated in the RAM model yielding faster Boolean matrix multiplication algorithms. Very recently, Chung et al [28] show, using techniques from the data structures and cryptography literature, that there exist functions whose computation, assuming the validity of a popular methodology for designing cryptographic constructions, is essentially not parallelizable in the MPC model.…”