Distributed Statistical Estimation of Matrix Products with Applications

Woodruff, David P.; Zhang, Qin

doi:10.1145/3196959.3196964

Cited by 7 publications

(5 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Namely, nearly all lower bounds for the space complexity of randomized streaming algorithms are derived via reductions from communication problems. For an incomplete list of such reductions, see [58,61,45,42,46,10,16,57,48,49,40] and the references therein. Now nearly all such lower bounds (and all of the ones that were just cited) hold in either the 2-party setting (G has 2 vertices), the coordinator model, or the black-board model.…”

Section: Multi-party Communicationmentioning

confidence: 99%

“…The black-board model is also considered frequently for designing communication upper bounds, such as those for set disjointness [6,16,30]. Finally, there is substantial literature which considers numerical linear algebra and clustering problems in the coordinator model [61,20,5,62]. Thus, our upper bounds can be seen as a new and useful contribution to these bodies of literature as well.…”

Section: Multi-party Communicationmentioning

confidence: 99%

See 1 more Smart Citation

Towards Optimal Moment Estimation in Streaming and Distributed Models

Jayaram

Woodruff

2023

ACM Trans. Algorithms

Self Cite

View full text Add to dashboard Cite

One of the oldest problems in the data stream model is to approximate the p -th moment \(\Vert \mathbf {X}\Vert _p^p = \sum _{i=1}^n \mathbf {X}_i^p \) of an underlying non-negative vector \(\mathbf {X} \in \mathbb {R}^n \) , which is presented as a sequence of poly( n ) updates to its coordinates. Of particular interest is when p ∈ (0, 2]. Although a tight space bound of Θ (ϵ − 2 log n ) bits is known for this problem when both positive and negative updates are allowed, surprisingly there is still a gap in the space complexity of this problem when all updates are positive. Specifically, the upper bound is O (ϵ − 2 log n ) bits, while the lower bound is only Ω (ϵ − 2 + log n ) bits. Recently, an upper bound of \(\tilde{O}(\epsilon ^{-2} + \log n) \) bits was obtained under the assumption that the updates arrive in a random order . We show that for p ∈ (0, 1], the random order assumption is not needed. Namely, we give an upper bound for worst-case streams of \(\tilde{O}(\epsilon ^{-2} + \log n) \) bits for estimating \(\Vert \mathbf {X}\Vert _p^p \) . Our techniques also give new upper bounds for estimating the empirical entropy in a stream. On the other hand, we show that for p ∈ (1, 2], in the natural coordinator and blackboard distributed communication topologies, there is an \(\tilde{O}(\epsilon ^{-2}) \) bit max-communication upper bound based on a randomized rounding scheme. Our protocols also give rise to protocols for heavy hitters and approximate matrix product. We generalize our results to arbitrary communication topologies G , obtaining an \(\tilde{O}(\epsilon ^{2} \log d) \) max-communication upper bound, where d is the diameter of G . Interestingly, our upper bound rules out natural communication complexity-based approaches for proving an Ω (ϵ − 2 log n ) bit lower bound for p ∈ (1, 2] for streaming algorithms. In particular, any such lower bound must come from a topology with large diameter.

show abstract

Section: Multi-party Communicationmentioning

confidence: 99%

Section: Multi-party Communicationmentioning

confidence: 99%

Towards Optimal Moment Estimation in Streaming and Distributed Models

Jayaram

Woodruff

2023

ACM Trans. Algorithms

Self Cite

View full text Add to dashboard Cite

show abstract

“…Statistics of a matrix product. In [45], an algorithm was given for estimating \| A \cdot B\| p for integer matrices A and B with O(log n) bit integer entries (see Algorithm 1 in [45] for the general algorithm). When p = 0, this estimates the number of nonzero entries of A \cdot B, which may be useful since there are faster algorithms for matrix product when the output is sparse; see [34] and the references therein.…”

Section: Problemmentioning

confidence: 99%

“…More generally, norms of the product A \cdot B can be used to determine how correlated the rows of A are with the columns of B. The bit complexity of this problem was studied in [42,45]. In [42] a lower bound of \Omega (\varepsi - 2 n) bits was shown for estimating \| AB\| 0 for n \times n matrices A, B up to a (1 + \varepsi ) factor, assuming n \geq 1/\varepsi 2 (this lower bound holds already for binary matrices A and B).…”

Section: Problemmentioning

confidence: 99%

“…Our lower bound in Theorem 1.2 considerably strengthens this result by showing the same lower bound (up to polylog(d/\varepsi ) factors) for a much smaller value of d = \Omega (log(1/\varepsi )). For any p \in [0, 2], there is a matching upper bound up to polylogarithmic factors (such an upper bound is given implicitly in the description of Algorithm 1 of [45], where the \varepsi there is instantiated with \surd \varepsi , and also follows from the random sketching matrices S discussed above). Downloaded 12/07/21 to 3.1.58.13 Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/page/terms Projective clustering.…”

Section: Problemmentioning

confidence: 99%

See 1 more Smart Citation

Tight Bounds for the Subspace Sketch Problem with Applications

Li¹,

Wang²,

Woodruff³

2021

SIAM J. Comput.

Self Cite

View full text Add to dashboard Cite

In the subspace sketch problem one is given an n \times d matrix A with O(log(nd)) bit entries, and would like to compress it in an arbitrary way to build a small space data structure Qp, so that for any given x \in \BbbR d , with probability at least 2/3, one has Qp(x) = (1 \pm \varepsi )\| Ax\| p, where p \geq 0 and the randomness is over the construction of Qp. The central question is, how many bits are necessary to store Qp? This problem has applications to the communication of approximating the number of nonzeros in a matrix product, the size of coresets in projective clustering, the memory of streaming algorithms for regression in the row-update model, and embedding subspaces of Lp in functional analysis. A major open question is the dependence on the approximation factor \varepsi . We show if p \geq 0 is not a positive even integer and d = \Omega (log(1/\varepsi )), then \widetil \Omega (\varepsi - 2 d) bits are necessary. On the other hand, if p is a positive even integer, then there is an upper bound of O(d p log(nd)) bits independent of \varepsi . Our results are optimal up to logarithmic factors. As corollaries of our main lower bound, we obtain new lower bounds for a wide range of applications, including the above, which in many cases are optimal.

show abstract

A New Statistics Collecting Method with Adaptive Strategy

Gao

Liu

et al. 2019

Database Systems for Advanced Applications

View full text Add to dashboard Cite

Distributed Statistical Estimation of Matrix Products with Applications

Cited by 7 publications

References 42 publications

Towards Optimal Moment Estimation in Streaming and Distributed Models

Towards Optimal Moment Estimation in Streaming and Distributed Models

Tight Bounds for the Subspace Sketch Problem with Applications

A New Statistics Collecting Method with Adaptive Strategy

Contact Info

Product

Resources

About