Differentially Private Release and Learning of Threshold Functions

Abstract

We prove new upper and lower bounds on the sample complexity of (ε, δ) differentially private algorithms for releasing approximate answers to threshold functions. A threshold function c x over a totally ordered domain X evaluates to c x (y) = 1 if y ≤ x, and evaluates to 0 otherwise. We give the first nontrivial lower bound for releasing thresholds with (ε, δ) differential privacy, showing that the task is impossible over an infinite domain X, and moreover requires sample complexity n ≥ Ω(log * |X|), which grows with the size of the domain. Inspired by the techniques used to prove this lower bound, we give an algorithm for releasing thresholds with n ≤ 2 (1+o(1)) log * |X| samples. This improves the previous best upper bound of 8(Beimel et al., RANDOM '13). Our sample complexity upper and lower bounds also apply to the tasks of learning distributions with respect to Kolmogorov distance and of properly PAC learning thresholds with differential privacy. The lower bound gives the first separation between the sample complexity of properly learning a concept class with (ε, δ) differential privacy and learning without privacy. For properly learning thresholds in dimensions, this lower bound extends to n ≥ Ω( ·log * |X|). To obtain our results, we give reductions in both directions from releasing and properly learning thresholds and the simpler interior point problem. Given a database D of elements from X, the interior point problem asks for an element between the smallest and largest elements in D. We introduce new recursive constructions for bounding the sample complexity of the interior point problem, as well as further reductions and techniques for proving impossibility results for other basic problems in differential privacy.

Make Up Your Mind: The Price of Online Queries in Differential Privacy

“…, τ k , 1} without changing the answers to any of the queries. Then we can use known algorithms for answering all threshold queries over any finite, totally ordered domain [2,5] to answer the queries using a very small dataset of size n = 2 O(log * (k)) . We leave it as an interesting open question to settle the complexity of answering threshold queries in the adaptive model.…”

“…[Proof of Theorem 5.16] Our algorithm and its analysis follow the reduction of Bun et al [5] for reducing the (offline) query release problem for thresholds to the offline interior point problem. Let T be an (ε, δ)-differentially private algorithm solving the AIP Problem with confidence αβ/8 and sample complexity n .…”

“…Once we have this algorithm, we can use it to answer adaptively-chosen thresholds using an approach inspired by Bun et al [5]. The high-level ideal is to sort the dataset x (1) < x (2) < .…”

Shortest Paths and Distances with Differential Privacy

“…In addition to the work of Dwork et al [DNPR10] described above, additional bounds and reductions for query release and learning of threshold functions are shown in Bun et al [BNSV15].…”

Order-Revealing Encryption and the Hardness of Private Learning

“…Since then, a number of works have improved our understanding of the sample complexity -the minimum number of examples -required by such learners to simultaneously achieve accuracy and privacy. Some of these works showed that privacy incurs an inherent additional cost in sample complexity; that is, some concept classes require more samples to learn privately than they require to learn without privacy [BKN10,CH11,BNS13,FX14,CHS14,BNSV15]. In this work, we address the complementary question of whether there is also a computational price of differential privacy for learning tasks, for which much less is known.…”