Simultaneous Private Learning of Multiple Concepts

Bun, Mark; Nissim, Kobbi; Stemmer, Uri

doi:10.1145/2840728.2840747

Cited by 51 publications

(70 citation statements)

References 35 publications

(96 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our analysis allows to bound (ε, δ)-DP directly and only requires increasing σ by a factor of max{Õ(k/n), 1}. We note that in the context of PAC learning sample complexity of solving multiple learning problems with differential privacy was studied in [BNS16b]. The question of optimizing multiple loss functions was also studied in [Ull15,FGV17].…”

Section: Multiple Convex Optimizationsmentioning

confidence: 99%

Privacy Amplification by Iteration

Feldman¹,

Mironov²,

Talwar³

et al. 2018

2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS)

172

View full text Add to dashboard Cite

Many commonly used learning algorithms work by iteratively updating an intermediate solution using one or a few data points in each iteration. Analysis of differential privacy for such algorithms often involves ensuring privacy of each step and then reasoning about the cumulative privacy cost of the algorithm. This is enabled by composition theorems for differential privacy that allow releasing of all the intermediate results. In this work, we demonstrate that for contractive iterations, not releasing the intermediate results strongly amplifies the privacy guarantees.We describe several applications of this new analysis technique to solving convex optimization problems via noisy stochastic gradient descent. For example, we demonstrate that a relatively small number of non-private data points from the same distribution can be used to close the gap between private and non-private convex optimization. In addition, we demonstrate that we can achieve guarantees similar to those obtainable using the privacy-amplification-by-sampling technique in several natural settings where that technique cannot be applied.

show abstract

Section: Multiple Convex Optimizationsmentioning

confidence: 99%

Privacy Amplification by Iteration

Feldman¹,

Mironov²,

Talwar³

et al. 2018

2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS)

172

View full text Add to dashboard Cite

show abstract

“…We note that the Ramsey argument in the first part is quite general: it does not use the definition of differential privacy and could perhaps be useful in other sample complexity lower bounds. Also, a similar argument was used by Bun [2016] in a weaker lower bound for privately learning thresholds in the proper case. However, the second and more technical part of the proof is tailored specifically to the definition of differential privacy.…”

Section: Proof Overviewmentioning

confidence: 84%

Private PAC learning implies finite Littlestone dimension

Alon

Livni

Malliaris

et al. 2019

Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

We show that every approximately differentially private learning algorithm (possibly improper) for a class H with Littlestone dimension d requires Ω log * (d) examples. As a corollary it follows that the class of thresholds over N can not be learned in a private manner; this resolves open questions due to [Bun et al., 2015, Feldman andXiao, 2015]. We leave as an open question whether every class with a finite Littlestone dimension can be learned by an approximately differentially private algorithm.

show abstract

“…For a large data universe X , the at least linear in |X | running time of BasicHistogram M,X can be prohibitive. By using approximate differential privacy, we can release counts for a smaller number of bins (at most n) based on stability techniques [KKMN09,BNS16]. We present a generalization of the algorithm from [BNS16].…”

Section: Stability-based Histogrammentioning

confidence: 99%

Differential Privacy on Finite Computers

Balcer

Vadhan

2019

JPC

View full text Add to dashboard Cite

We consider the problem of designing and analyzing differentially private algorithms that can be implemented on discrete models of computation in strict polynomial time, motivated by known attacks on floating point implementations of real-arithmetic differentially private algorithms (Mironov, CCS 2012) and the potential for timing attacks on expected polynomialtime algorithms. As a case study, we examine the basic problem of approximating the histogram of a categorical dataset over a possibly large data universe X . The classic Laplace Mechanism (Dwork, McSherry, Nissim, Smith, TCC 2006 and J. Privacy & Confidentiality 2017) does not satisfy our requirements, as it is based on real arithmetic, and natural discrete analogues, such as the Geometric Mechanism (Ghosh, Roughgarden, Sundarajan, STOC 2009 and SICOMP 2012), take time at least linear in |X |, which can be exponential in the bit length of the input.In this paper, we provide strict polynomial-time discrete algorithms for approximate histograms whose simultaneous accuracy (the maximum error over all bins) matches that of the Laplace Mechanism up to constant factors, while retaining the same (pure) differential privacy guarantee. One of our algorithms produces a sparse histogram as output. Its "per-bin accuracy" (the error on individual bins) is worse than that of the Laplace Mechanism by a factor of log |X |, but we prove a lower bound showing that this is necessary for any algorithm that produces a sparse histogram. A second algorithm avoids this lower bound, and matches the per-bin accuracy of the Laplace Mechanism, by producing a compact and efficiently computable representation of a dense histogram; it is based on an (n + 1)-wise independent implementation of an appropriately clamped version of the Discrete Geometric Mechanism. * A condensed version of this paper appeared in ITCS 2018 [BV18] †

show abstract

Simultaneous Private Learning of Multiple Concepts

Cited by 51 publications

References 35 publications

Privacy Amplification by Iteration

Privacy Amplification by Iteration

Private PAC learning implies finite Littlestone dimension

Differential Privacy on Finite Computers

Contact Info

Product

Resources

About