Fooling views: a new lower bound technique for distributed computations under congestion

Abboud, Amir; Censor-Hillel, Keren; Khoury, Seri; Lenzen, Christoph

doi:10.1007/s00446-020-00373-4

Cited by 16 publications

(56 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Whether there are faster algorithms for triangle detection 10 is an intriguing open problem. It is known that 1-round LOCAL algorithms must send messages of Ω(∆ log n) bits deterministically [ACKL17] or Ω(∆) bits randomized [FGKO18]. Even for 2-round triangle detection algorithms, there are no nontrivial communication lower bounds known.…”

Section: Conclusion and Open Problemsmentioning

confidence: 99%

The Communication Complexity of Set Intersection and Multiple Equality Testing

Huang¹,

Pettie²,

Zhang³

et al. 2020

Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

In this paper we explore fundamental problems in randomized communication complexity such as computing Set Intersection on sets of size k and Equality Testing between vectors of length k. Brody et al. [BCK + 16] and Sağlam and Tardos [ST13] showed that for these types of problems, one can achieve optimal communication volume of O(k) bits, with a randomized protocol that takes O(log * k) rounds. They also proved [BCK + 16, ST13] that this is one point along the optimal round-communication tradeoff curve.Aside from rounds and communication volume, there is a third parameter of interest, namely the error probability p err . It is straightforward to show that protocols for Set Intersection or Equality Testing need to send Ω(k + log p −1 err ) bits. Is it possible to simultaneously achieve optimality in all three parameters, namely O(k + log p −1 err ) communication and O(log * k) rounds? In this paper we prove that there is no universally optimal algorithm, and complement the existing round-communication tradeoffs [BCK + 16, ST13] with a new tradeoff between rounds, communication, and probability of error. In particular: * 1 Communication Complexity was defined by Yao [Yao79] in 1979 and has become an indispensible tool for proving lower bounds in models of computation in which the notions of parties and communication are not direct. See, e.g., books and monographs [Rou16, RY, KN97] and surveys [CP10, Lov89] on the subject. In this paper we consider some of the most fundamental and well-studied problems in this model, such as SetDisjointness, SetIntersection, ExistsEqual, and EqualityTesting. Let us briefly define these problems formally since the terminology is not completely standard.SetDisjointness and SetIntersection. In the SetDisjointness problem Alice and Bob receive sets A ⊂ U and B ⊂ U where |A|, |B| ≤ k and must determine whether A ∩ B = ∅. Define SetDisj(k, r, p err ) to be the minimum communication complexity of an r-round randomized protocol for this problem that errs with probability at most p err . We can assume that |U | = O(k 2 /p err ) without loss of generality. 1 The input to the SetIntersection problem is the same, except that the parties must report the entire set A ∩ B. Define SetInt(k, r, p err ) to be the minimum communication complexity of an r-round protocol for SetIntersection. EqualityTesting and ExistsEqual. In the EqualityTesting problem Alice and Bob hold vectors x ∈ U k and y ∈ U k and must determine, for each index i ∈ [k], whether x i = y i or x i = y i . A potentially easier version of the problem, ExistsEqual, is to determine if there exists at least one index i ∈ [k] for which x i = y i . Define Eq(k, r, p err ) to be the randomized communication complexity of any r-round protocol for EqualityTesting that errs with probability p err , and ∃Eq(k, r, p err ) the corresponding complexity of ExistsEqual. Once again, we can assume that |U | = O(k/p err ) without loss of generality. The deterministic communication complexity of these problems is well understood [KN97], 2 so we consid...

show abstract

Section: Conclusion and Open Problemsmentioning

confidence: 99%

The Communication Complexity of Set Intersection and Multiple Equality Testing

Huang¹,

Pettie²,

Zhang³

et al. 2020

Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

show abstract

“…There have been recent attempts to combine the edge crossing and bottleneck techniques to obtain lower bounds for triangle detection in the Congest model [Abb+17;Fis+18]. In particular, [Fis+18] provide an Ω(log n) lower bound for deterministic algorithms solving triangle detection in the KT-1 Congest model with 1-bit bandwidth.…”

Section: Related Workmentioning

confidence: 99%

Connectivity Lower Bounds in Broadcast Congested Clique

Pai

Pemmaraju

2019

Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

We prove three new lower bounds for graph connectivity in the 1-bit broadcast congested clique model, BCC(1). First, in the KT-0 version of BCC(1), in which nodes are aware of neighbors only through port numbers, we show an Ω(log n) round lower bound for Connectivity even for constant-error randomized Monte Carlo algorithms. The deterministic version of this result can be obtained via the well-known "edge-crossing" argument, but, the randomized version of this result requires establishing new combinatorial results regarding the indistinguishability graph induced by inputs. In our second result, we show that the Ω(log n) lower bound result extends to the KT-1 version of the BCC(1) model, in which nodes are aware of IDs of all neighbors, though our proof works only for deterministic algorithms. Since nodes know IDs of their neighbors in the KT-1 model, it is no longer possible to play "edge-crossing" tricks; instead we present a reduction from the 2-party communication complexity problem Partition in which Alice and Bob are given two set partitions on [n] and are required to determine if the join of these two set partitions equals the trivial one-part set partition. While our KT-1 Connectivity lower bound holds only for deterministic algorithms, in our third result we extend this Ω(log n) KT-1 lower bound to constant-error Monte Carlo algorithms for the closely related ConnectedComponents problem. We use information-theoretic techniques to obtain this result. All our results hold for the seemingly easy special case of Connectivity in which an algorithm has to distinguish an instance with one cycle from an instance with multiple cycles. Our results showcase three rather different lower bound techniques and lay the groundwork for further improvements in lower bounds for Connectivity in the BCC(1) model. * A short version of this paper has appeared as a brief announcement in PODC 2019. 1 We use "w.h.p." as short for "with high probability" which refers to the probability that is at least 1 − 1/n c for c ≥ 1. arXiv:1905.09016v1 [cs.DC] 22 May 2019 for Connectivity in the BCC(log n) model, due to Jurdziński and Nowicki [JN17], is deterministic and it runs in O log n log log n rounds. This contrast between BCC(b) and CC(b) is not surprising, given how much larger the overall bandwidth in CC(b) is compared to BCC(b). Becker et al. [Bec+16]show that the pair-wise set disjointness problem can be solved in O(1) rounds in CC (1), but needs Ω(n) rounds in BCC(1). But, despite the fact that Connectivity is such a fundamental problem, no non-trivial lower bound is known for Connectivity in BCC(1). In fact, prior to this paper, we could not even rule out an O(1)-round Connectivity algorithm in BCC(1).Lower bound arguments in "congested" distributed computing models typically use a "bottleneck" technique [CKP17; CK18; DS+11; DKO14; Fis+18; HP15]. At a high level, this technique consists of showing that there is a low bandwidth cut in the communication network across which a high volume of information has to flow in order to solve...

show abstract

“…If the output cut is S, then the time spent isÕ(Vol(S)). 1 By iteratively finding a sparse cut and removing it from the graph, inÕ(|E|) time a graph partition is obtained in which all components have Ω(1/polylog(n)) conductance.…”

Section: Introductionmentioning

confidence: 99%

Distributed Triangle Detection via Expander Decomposition

Chang

Pettie

Zhang

2019

Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

We present improved distributed algorithms for triangle detection and its variants in the CONGEST model. We show that Triangle Detection, Counting, and Enumeration can be solved inÕ(n 1/2 ) rounds. In contrast, the previous state-of-the-art bounds for Triangle Detection and Enumeration wereÕ(n 2/3 ) andÕ(n 3/4 ), respectively, due to Izumi and LeGall (PODC 2017).The main technical novelty in this work is a distributed graph partitioning algorithm. We show that inÕ(n 1−δ ) rounds we can partition the edge set of the network G• Each connected component induced by E m has minimum degree Ω(n δ ) and conductance Ω(1/polylog(n)). As a consequence the mixing time of a random walk within the component is O(polylog(n)).• The subgraph induced by E s has arboricity at most n δ .• |E r | ≤ |E|/6.All of our algorithms are based on the following generic framework, which we believe is of interest beyond this work. Roughly, we deal with the set E s by an algorithm that is efficient for low-arboricity graphs, and deal with the set E r using recursive calls. For each connected component induced by E m , we are able to simulate CONGESTED-CLIQUE algorithms with small overhead by applying a routing algorithm due to Ghaffari, Kuhn, and Su (PODC 2017) for high conductance graphs.We consider Triangle Detection problems in distributed networks. In the LOCAL model [34], which has no limit on bandwidth, all variants of Triangle Detection can be solved in exactly one round of communication: every vertex v simply announces its neighborhood N (v) to all neighbors. However, in models that take bandwidth into account, e.g., CONGEST, Triangle Detection becomes significantly more complicated. Whereas many graph optimization problems studied in the CONGEST model are intrinsically "global" (i.e., require at least diameter time) [2,11,12,14,15,20,26], Triangle Detection is somewhat unusual in that it can, in principle, be solved using only locally available information.The CONGEST Model. The underlying distributed network is represented as an undirected graph G = (V, E), where each vertex corresponds to a computational device, and each edge corresponds to a bi-directional communication link. We assume each v ∈ V initially knows some global parameters such as n = |V |, ∆ = max v∈V deg(v), and D = diameter(G). Each vertex v has a distinct Θ(log n)-bit identifier ID(v). The computation proceeds according to synchronized rounds.In each round, each vertex v can perform unlimited local computation, and may send a distinct O(log n)-bit message to each of its neighbors. Throughout the paper we only consider the randomized variant of CONGEST. Each vertex is allowed to generate unlimited local random bits, but there is no global randomness.The Congested Clique Model. The CONGESTED-CLIQUE model is a variant of CONGEST that allows all-to-all communication. Each vertex initially knows its adjacent edges and the set of vertex IDs, which we can assume w.l.o.g. is {1, . . . , |V |}. In each round, each vertex transmits n − 1 O(log n)-bit messages, one addressed to ...

show abstract

Fooling views: a new lower bound technique for distributed computations under congestion

Cited by 16 publications

References 74 publications

The Communication Complexity of Set Intersection and Multiple Equality Testing

The Communication Complexity of Set Intersection and Multiple Equality Testing

Connectivity Lower Bounds in Broadcast Congested Clique

Distributed Triangle Detection via Expander Decomposition

Contact Info

Product

Resources

About