The Communication Complexity of Set Intersection and Multiple Equality Testing

Abstract

In this paper we explore fundamental problems in randomized communication complexity such as computing Set Intersection on sets of size k and Equality Testing between vectors of length k. Brody et al. [BCK + 16] and Sağlam and Tardos [ST13] showed that for these types of problems, one can achieve optimal communication volume of O(k) bits, with a randomized protocol that takes O(log * k) rounds. They also proved [BCK + 16, ST13] that this is one point along the optimal round-communication tradeoff curve.Aside from rounds and communication volume, there is a third parameter of interest, namely the error probability p err . It is straightforward to show that protocols for Set Intersection or Equality Testing need to send Ω(k + log p −1 err ) bits. Is it possible to simultaneously achieve optimality in all three parameters, namely O(k + log p −1 err ) communication and O(log * k) rounds? In this paper we prove that there is no universally optimal algorithm, and complement the existing round-communication tradeoffs [BCK + 16, ST13] with a new tradeoff between rounds, communication, and probability of error. In particular: * 1 Communication Complexity was defined by Yao [Yao79] in 1979 and has become an indispensible tool for proving lower bounds in models of computation in which the notions of parties and communication are not direct. See, e.g., books and monographs [Rou16, RY, KN97] and surveys [CP10, Lov89] on the subject. In this paper we consider some of the most fundamental and well-studied problems in this model, such as SetDisjointness, SetIntersection, ExistsEqual, and EqualityTesting. Let us briefly define these problems formally since the terminology is not completely standard.SetDisjointness and SetIntersection. In the SetDisjointness problem Alice and Bob receive sets A ⊂ U and B ⊂ U where |A|, |B| ≤ k and must determine whether A ∩ B = ∅. Define SetDisj(k, r, p err ) to be the minimum communication complexity of an r-round randomized protocol for this problem that errs with probability at most p err . We can assume that |U | = O(k 2 /p err ) without loss of generality. 1 The input to the SetIntersection problem is the same, except that the parties must report the entire set A ∩ B. Define SetInt(k, r, p err ) to be the minimum communication complexity of an r-round protocol for SetIntersection. EqualityTesting and ExistsEqual. In the EqualityTesting problem Alice and Bob hold vectors x ∈ U k and y ∈ U k and must determine, for each index i ∈ [k], whether x i = y i or x i = y i . A potentially easier version of the problem, ExistsEqual, is to determine if there exists at least one index i ∈ [k] for which x i = y i . Define Eq(k, r, p err ) to be the randomized communication complexity of any r-round protocol for EqualityTesting that errs with probability p err , and ∃Eq(k, r, p err ) the corresponding complexity of ExistsEqual. Once again, we can assume that |U | = O(k/p err ) without loss of generality. The deterministic communication complexity of these problems is well understood [KN97], 2 so we consid...

Distributed Subgraph Finding: Progress and Challenges

“…An additional algorithm for deterministic triangle listing is given by Huang, Pettie, Zhang, and Zhang [HPZZ20]. The complexity of this algorithm, given in terms of the maximum degree ∆, is O(∆/ log n + log log ∆) rounds, w.h.p.…”

“…Since the complexities of both [CS20] and [HPZZ20] do not reach yet the lower bound of Ω(n 1/3 ) rounds, we note the following open question.…”

Deterministic Near-Optimal Distributed Listing of Cliques

“…Subsequently, [CPSZ21] showed a breakthrough by listing in the optimal complexity of ˜ ( 1/3 ) rounds, w.h.p., using an expander decomposition together with the routing techniques of [GKS17,GL18]. [HPZZ21] show an algorithm which takes (Δ/log + log log Δ) rounds, where Δ is the maximal degree in the graph. following: Subsequently, [CPZ19] showed a breakthrough by listing in ˜ ( 1/2 ) rounds, w.h.p., using an expander decomposition, which was later improved to get the optimal, yet randomized, ˜ ( 1/3 )-round algorithm of [CS19].…”

Deterministic Near-Optimal Distributed Listing of Cliques

“…Subsequently, [13] showed a breakthrough by listing in the optimal complexity of Õ (𝑛 1/3 ) rounds, w.h.p., using an expander decomposition together with the routing techniques of [25,26]. [29] show an 𝑂 (Δ/log 𝑛 + log log Δ)-round algorithm, where Δ is the maximal degree in the graph. The first non-trivial deterministic algorithm was given by [16], taking 𝑂 (𝑛 0.58 ) rounds for detection and 𝑛 2/3+𝑜 (1) for listing.…”