An Equivalence Between Private Classification and Online Prediction

Bun, Mark; Livni, Roi; Moran, Shay

doi:10.1109/focs46700.2020.00044

Cited by 34 publications

(47 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Theorem 6.2 then carries over verbatim to our notion of quantum SQ learnability. This form of quantum differential privacy was recently studied by Arunachalam et al [11], who were able to relate it to online learning, one-way communication complexity, and shadow tomography of quantum states, extending ideas of Bun et al [18]. Since our notion of quantum SQ learnability implies quantum DP learnability, it also fits into their framework.…”

Section: Connections To Differential Privacymentioning

confidence: 79%

“…Recent work by Arunachalam et al [11] extends work by Bun et al [18] to the quantum setting, and relates differentially private (DP) learning of quantum states to one-way communication, online learning, and other models. We show in Section 6 that our notion of SQ learnability implies their notion of DP learnability, and hence by their results also implies finite sequential fat-shattering dimension, online learnability, and "quantum stability.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On the Hardness of PAC-learning Stabilizer States with Noise

Gollakota

Liang

2022

Quantum

View full text Add to dashboard Cite

We consider the problem of learning stabilizer states with noise in the Probably Approximately Correct (PAC) framework of Aaronson (2007) for learning quantum states. In the noiseless setting, an algorithm for this problem was recently given by Rocchetto (2018), but the noisy case was left open. Motivated by approaches to noise tolerance from classical learning theory, we introduce the Statistical Query (SQ) model for PAC-learning quantum states, and prove that algorithms in this model are indeed resilient to common forms of noise, including classification and depolarizing noise. We prove an exponential lower bound on learning stabilizer states in the SQ model. Even outside the SQ model, we prove that learning stabilizer states with noise is in general as hard as Learning Parity with Noise (LPN) using classical examples. Our results position the problem of learning stabilizer states as a natural quantum analogue of the classical problem of learning parities: easy in the noiseless setting, but seemingly intractable even with simple forms of noise.

show abstract

Section: Connections To Differential Privacymentioning

confidence: 79%

Section: Related Workmentioning

confidence: 99%

On the Hardness of PAC-learning Stabilizer States with Noise

Gollakota

Liang

2022

Quantum

View full text Add to dashboard Cite

show abstract

“…This assumption is quite strong, but not completely unreasonable. Specifically, Bun et al [BLM20]in their seminal work that characterizes hypothesis classes learnable in the item-level DP settingshowed that it is possible to come up with a learner that outputs some hypothesis h * with probability 2 −O(d) , where d denotes the Littlestone dimension of the concept class. We may attempt to use this in the approach described above, but this does not work: in order to even see h * at all (with say a constant probability), we would need 2 O(d) users, which is prohibitive!…”

Section: Proof Overviewmentioning

confidence: 99%

“…In this paper we address the question in a very general setting: what can user-level privacy gain for any privately PAC learnable class? Recall that it had recently been shown that a class is learnable via (ε, δ)-DP algorithms iff it is online learnable [ALMM19,BLM20], which is in turn equivalent to the class having a finite Littlestone dimension [Lit87]. Furthermore, it is also known that a class is learnable via ε-DP algorithms iff it has a finite probabilistic dimension [BNS19a].…”

Section: Introductionmentioning

confidence: 99%

User-Level Private Learning via Correlated Sampling

Ghazi¹,

Kumar²,

Manurangsi³

2021

Preprint

View full text Add to dashboard Cite

Most works in learning with differential privacy (DP) have focused on the setting where each user has a single sample. In this work, we consider the setting where each user holds m samples and the privacy protection is enforced at the level of each user's data. We show that, in this setting, we may learn with a much fewer number of users. Specifically, we show that, as long as each user receives sufficiently many samples, we can learn any privately learnable class via an (ε, δ)-DP algorithm using only O(log(1/δ)/ε) users. For ε-DP algorithms, we show that we can learn using only O ε (d) users even in the local model, where d is the probabilistic representation dimension. In both cases, we show a nearly-matching lower bound on the number of users required.A crucial component of our results is a generalization of global stability [BLM20] that allows the use of public randomness. Under this relaxed notion, we employ a correlated sampling strategy to show that the global stability can be boosted to be arbitrarily close to one, at a polynomial expense in the number of samples.

show abstract

“…In the setting of differentially private binary classification, a major recent development [ALMM19,BLM20] is the result that a hypothesis class F consisting of binary classifiers is learnable with approximate differential privacy (Definition 2.1) if and only if it is online learnable, which is known to hold in turn if and only if the Littlestone dimension of F is finite [Lit87, BPS09]. Such an equivalence, however, remains open for the setting of differentially private regression (this question was asked in [BLM20]). The combinatorial parameter characterizing online learnability for regression is the sequential fat-shattering dimension [RST15b] (Definition 2.4), which may be viewed as a scale-sensitive analogue of the Littlestone dimension.…”

Section: Introductionmentioning

confidence: 99%

Differentially Private Nonparametric Regression Under a Growth Condition

Golowich¹

2021

Preprint

View full text Add to dashboard Cite

Given a real-valued hypothesis class H, we investigate under what conditions there is a differentially private algorithm which learns an optimal hypothesis from H given i.i.d. data. Inspired by recent results for the related setting of binary classification [ALMM19, BLM20], where it was shown that online learnability of a binary class is necessary and sufficient for its private learnability, [JKT20] showed that in the setting of regression, online learnability of H is necessary for private learnability. Here online learnability of H is characterized by the finiteness of its η-sequential fat shattering dimension, sfat η (H), for all η > 0. In terms of sufficient conditions for private learnability, [JKT20] showed that H is privately learnable if lim η↓0 sfat η (H) is finite, which is a fairly restrictive condition. We show that under the relaxed condition lim inf η↓0 η • sfat η (H) = 0, H is privately learnable, establishing the first nonparametric private learnability guarantee for classes H with sfat η (H) diverging as η ↓ 0. Our techniques involve a novel filtering procedure to output stable hypotheses for nonparametric function classes.

show abstract

An Equivalence Between Private Classification and Online Prediction

Cited by 34 publications

References 26 publications

On the Hardness of PAC-learning Stabilizer States with Noise

On the Hardness of PAC-learning Stabilizer States with Noise

User-Level Private Learning via Correlated Sampling

Differentially Private Nonparametric Regression Under a Growth Condition

Contact Info

Product

Resources

About