A New Approach for Testing Properties of Discrete Distributions

Abstract

We study problems in distribution property testing: Given sample access to one or more unknown discrete distributions, we want to determine whether they have some global property or are ǫ-far from having the property in ℓ 1 distance (equivalently, total variation distance, or "statistical distance"). In this work, we give a novel general approach for distribution testing. We describe two techniques: our first technique gives sample-optimal testers, while our second technique gives matching sample lower bounds. As a consequence, we resolve the sample complexity of a wide variety of testing problems.Our upper bounds are obtained via a modular reduction-based approach. Our approach yields optimal testers for numerous problems by using a standard ℓ 2 -identity tester as a blackbox. Using this recipe, we obtain simple estimators for a wide range of problems, encompassing most problems previously studied in the TCS literature, namely: (1) identity testing to a fixed distribution, (2) closeness testing between two unknown distributions (with equal/unequal sample sizes), (3) independence testing (in any number of dimensions), (4) closeness testing for collections of distributions, and (5) testing histograms. For all of these problems, our testers are sample-optimal, up to constant factors. With the exception of (1), ours are the first sampleoptimal testers for the corresponding problems. Moreover, our estimators are significantly simpler to state and analyze compared to previous results.As an important application of our reduction-based technique, we obtain the first nearly instance-optimal algorithm for testing equivalence between two unknown distributions. The sample complexity of our algorithm depends on the structure of the unknown distributions -as opposed to merely their domain size -and is much better compared to the worst-case optimal ℓ 1 -tester in most natural instances. Moreover, our technique naturally generalizes to other metrics beyond the ℓ 1 -distance. As an illustration of its flexibility, we use it to obtain the first near-optimal equivalence tester under the Hellinger distance.Our lower bounds are obtained via a direct information-theoretic approach: Given a candidate hard instance, our proof proceeds by bounding the mutual information between appropriate random variables. While this is a classical method in information theory, prior to our work, it had not been used in distribution property testing. Previous lower bounds relied either on the birthday paradox, or on moment-matching and were thus restricted to symmetric properties. Our lower bound approach does not suffer from any such restrictions and gives tight sample lower bounds for the aforementioned problems.

Which Distribution Distances are Sublinearly Testable?

“…This problem was also studied in the setting where one has unequal sample sizes from the two distributions [BV15,DK16]. When the distance d 1 is Hellinger, the complexity is qualitatively different, as shown by [DK16]. They prove a nearlyoptimal upper bound and a tight lower bound for this problem.…”

“…Tight upper and lower bounds were given in [CDVV14], which shows interesting behavior of the sample complexity as the parameter ε goes from large to small. This problem was also studied in the setting where one has unequal sample sizes from the two distributions [BV15,DK16]. When the distance d 1 is Hellinger, the complexity is qualitatively different, as shown by [DK16].…”

“…To address these challenges, there has been a surge of recent interest in information theory, property testing, and sublinear-time algorithms aiming at finite sample and d 1 -close vs. d 2 -far distinguishers, as in the formulations above; see e.g. [BFF + 01, BKR04, Pan08, VV17, ADK15, CDGR16,DK16]. This line of work has culminated in computationally efficient and sample optimal testers for several choices of distances d 1 and d 2 , as well as error parameters ε 1 and ε 2 , for example:…”

“…This can be seen by noting that the underlying tester from [CDVV14] is tolerant, and the "flattening" operation they apply reduces the 2 -distance between the distributions. The testers in [DK16] are those of Propositions 2.7, 2.10, and 2.15, combined with the observation of Remark 2.8. We rederive these results for completeness, and to show a direct way of proving 2 -tolerance.…”

“…In our identity tester for Hellinger, we deal with this different distance measure by pruning light domain elements of q less aggressively than [ADK15], in combination with a preliminary test to reject early if the difference between p and q is contained exclusively within the set of light elements -this is a new issue that cannot arise when testing with respect to total variation distance. In our equivalence tester for Hellinger, we follow an approach, similar to [CDVV14] and [DK16], of analyzing the light and heavy domain elements separately, with the challenge that the algorithm does not know which elements are which. Finally, to achieve 2 tolerance in these cases, we use a "mixing" strategy in which instead of testing based solely on samples from p and q, we mix in some number (depending on our application) of samples from the uniform distribution.…”