Sorting with adversarial comparators and application to density estimation

Abstract

We study maximum selection and sorting of n numbers using pairwise comparators that output the larger of their two inputs if the inputs are more than a given threshold apart, and output an adversariallychosen input otherwise. We consider two adversarial models. A non-adaptive adversary that decides on the outcomes in advance based solely on the inputs, and an adaptive adversary that can decide on the outcome of each query depending on previous queries and outcomes.Against the non-adaptive adversary, we derive a maximum-selection algorithm that uses at most 2n comparisons in expectation, and a sorting algorithm that uses at most 2n ln n comparisons in expectation. These numbers are within small constant factors from the best possible. Against the adaptive adversary, we propose a maximum-selection algorithm that uses Θ(n log(1/ǫ)) comparisons to output a correct answer with probability at least 1 − ǫ. The existence of this algorithm affirmatively resolves an open problem of Ajtai, Feldman, Hassadim, and Nelson [AFHN15].Our study was motivated by a density-estimation problem where, given samples from an unknown underlying distribution, we would like to find a distribution in a known class of n candidate distributions that is close to underlying distribution in ℓ1 distance. Scheffe's algorithm [DL01] outputs a distribution at an ℓ1 distance at most 9 times the minimum and runs in time Θ(n 2 log n). Using maximum selection, we propose an algorithm with the same approximation guarantee but run time of Θ(n log n). * Part of this paper appeared in [AJOS14].However, in many applications, the pairwise comparisons may be imprecise. For example, when comparing two random numbers, such as stock performances, or team strengths, the output of the comparison may vary due to chance. Consequently, a number of researchers have considered maximum selection and sorting with imperfect, or noisy, comparators.The comparators in these models mostly function correctly, but occasionally may produce an inaccurate comparison result, where the form of inaccuracy is dictated by the application. The Bradley-Terry-Luce [BT52] model assumes that if two values x and y are compared, then x is selected as the larger with probability x/(x + y). Observe that the comparison is correct with probability max{x, y}/(x + y) ≥ 1/2. Algorithms for ranking and estimating weights under this model were proposed, e.g., in [NOS12]. Another model assumes that the output of any comparator gets reversed with probability less than 1/2. Algorithms applying this model for maximum selection were proposed in [AGHB + 94] and for ranking in [KK07, BM08].We consider a third model where, unlike the previous models, the comparison outcome can be adversarial. If the numbers compared are more than a threshold ∆ apart, the comparison is correct, while if they differ by at most ∆, the comparison is arbitrary, and possibly even adversarial.This model can be partially motivated by physical observations. Measurements are regularly quantized and often adulterated with some measurement...

Sampling Correctors

“…We note that the quadratic running time can be brought down to near-linear, as shown in [DK14,AJOS14] (with the same sample complexity, although at the price of a worse semi-agnostic constant). This improved running time, however, is not crucial for our applications.…”

Multi-Item Mechanisms without Item-Independence: Learnability via Robustness

“…Theorem 12) provide broad generalizations of the bounds for learning Ising models and Gaussian MRFs presented in recent work of Devroye et al [29]. Their bounds are obtained by bounding the VC complexity of the Yatracos class induced by the distributions of interest, while our bounds are obtained by constructing ε-nets of the distributions of interest, and running a tournament-style hypothesis selection algorithm [28,25,1] to select one distribution from the net. Since the distribution families we consider are non-parametric, our main technical contribution is to bound the size of an ε-net sufficient to cover the distributions of interest.…”

“…While this recent work bounds the VC dimension of the Yatracos class of these families of distributions, for our more general families of non-parametric distributions we construct instead covers under either total variation distance or Prokhorov distance, and combine our cover-size upper bounds with generic tournament-style algorithms; see e.g. [28,25,1]. The details are provided in Appendix F. While there are many details, we illustrate one snippet of an idea used in constructing a ε-cover, in total variation distance, of the set of all MRFs with hyper-edges E of size at most d and a discrete alphabet Σ on every node.…”

“…This progress has enabled the identification of simple auctions (mostly variants of sequential posted pricing mechanisms) that achieve constant factor approximations to the optimum revenue [5,54,14,19,16], under item-independence assumptions. 1 Making Bayesian assumptions in the study of revenue-optimal auctions is both crucial and fruitful. However, to apply the theory to practice, we would need to know the underlying distributions.…”

On the Structure, Covering, and Learning of Poisson Multinomial Distributions

“…Besides providing an asymptotically smaller search space for Nash equilibria in anonymous games, or any other optimization problem over PMDs, the polynomial rather than exponential dependence of the cover size on n has direct consequences to the learnability of these distributions; see Theorem 7 (from [DK14]) and [AJOS14] for a similar result, which improve a long line of similar results in the probability literature [DL01]. In particular, a cover of polynomial size implies directly that these distributions can be learned from a number of samples logarithmic in n, despite their support being polynomial in n. Motivated by such applications of covers to algorithms and learning we use our structural result to obtain an improved cover theorem.…”