Inference under Information Constraints I: Lower Bounds from Chi-Square Contraction

Abstract

Multiple players are each given one independent sample, about which they can only provide limited information to a central referee. Each player is allowed to describe its observed sample to the referee using a channel from a family of channels W, which can be instantiated to capture both the communicationand privacy-constrained settings and beyond. The referee uses the messages from players to solve an inference problem for the unknown distribution that generated the samples. We derive lower bounds for sample complexity of learning and testing discrete distributions in this information-constrained setting.Underlying our bounds is a characterization of the contraction in chi-square distances between the observed distributions of the samples when information constraints are placed. This contraction is captured in a local neighborhood in terms of chi-square and decoupled chi-square fluctuations of a given channel, two quantities we introduce. The former captures the average distance between distributions of channel output for two product distributions on the input, and the latter for a product distribution and a mixture of product distribution on the input. Our bounds are tight for both public-and privatecoin protocols. Interestingly, the sample complexity of testing is order-wise higher when restricted to private-coin protocols.

Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters

“…This 1 risk is optimal over all protocols, even while allowing public-coins [26,32,24]. Note that compared to the centralized setting, the risk is a factor of Θ( √ k/ε) higher which shows the significant drop in the utility under LDP.…”

“…Distribution estimation has also been studied recently under very low communication budget [21,22,23,24], where each user sends only < log k bits to R. In particular, now it is established that by only using private-coin communication schemes,…”

“…For example, in mobile devices, and small sensors with a limited power/limited uplink capacity, the communication budget can overshadow the local computations performed at each of them. This has led to a growing interest in understanding various inference tasks under limited communication, where the users do not have enough communication to even transmit their data [19,20], and recent works have established optimal bounds, and algorithms for fundamental problems such as distribution estimation and hypothesis testing [21,22,23,24]. Like privacy, these works show that communication constraints also increase the data requirements for vatious tasks.…”

Optimal Communication Rates and Combinatorial Properties for Common Randomness Generation

“…We refer to [KN96] for a survey of these methods. Another closely-related problem is distributed inference under communication constraints [ZDJW13], where distributed simulation of private/public randomness is useful for distributed learning and property testing [ACT18a,ACT18b]. To establish lower bounds on the communication complexity in distributed inference, the copy-paste property of the blackboard communication model typically plays an important role [BGM + 16, HÖW18].…”

Limits on Gradient Compression for Stochastic Optimization

“…Also, SimQ + yields the exact upper bound in the ℓ 2 case. In the [1,2) range, we divide the vector into two parts with small and large coordinates. We use a uniform quantizer for the first part and RATQ of [14] for the second part.…”