Optimal Testing for Properties of Distributions

Abstract

Given samples from an unknown distribution p, is it possible to distinguish whether p belongs to some class of distributions C versus p being far from every distribution in C? This fundamental question has received tremendous attention in statistics, focusing primarily on asymptotic analysis, and more recently in information theory and theoretical computer science, where the emphasis has been on small sample size and computational complexity. Nevertheless, even for basic properties of distributions such as monotonicity, log-concavity, unimodality, independence, and monotone-hazard rate, the optimal sample complexity is unknown.We provide a general approach via which we obtain sample-optimal and computationally efficient testers for all these distribution families. At the core of our approach is an algorithm which solves the following problem: Given samples from an unknown distribution p, and a known distribution q, are p and q close in χ 2 -distance, or far in total variation distance? With this tool in place, we develop a general testing framework which leads to the following results: * Supported by a grant from MITEI-Shell program.

Distribution-free inference for regression: discrete, continuous, and in between

“…While the results above establish the challenges for distribution-free inference when the features X are nonatomic, at the other extreme we can consider scenarios where X has a discrete distribution. In this setting, the problem of estimating µ P is related to the discrete distribution testing, where the aim is to test properties of a discrete distribution-for instance, we might wish to test equality of two distributions where we draw samples from each [Chan et al, 2014, Acharya et al, 2014, Diakonikolas and Kane, 2016, Canonne et al, 2015; to test whether a sample is drawn from a known distribution P or not [Diakonikolas and Kane, 2016, Acharya et al, 2015, Valiant and Valiant, 2017, or drawn from any distribution belonging to a class P or not [Acharya et al, 2015, Canonne et al, 2018; or to estimate certain characteristics of a distribution based on a sample, such as its entropy or its support size [Valiant andValiant, 2011b,a, Acharya et al, 2014]. The methods developed in this literature are closely related to the problem of distribution-free confidence intervals, as we will see later on when we construct upper bounds in Section 3.…”