An Equivalence Between Private Classification and Online Prediction

Bun, Mark; Livni, Roi; Moran, Shay

doi:10.48550/arxiv.2003.00563

Cited by 5 publications

(39 citation statements)

References 34 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…non-trivial consequences (which we highlight later), one of which is that a technique due to [BLM20] showing stability implies approximate DP PAC for Boolean functions, is a no-go in our setting, as we show later.…”

Section: No-gomentioning

confidence: 66%

“…Such a connection has remained unexplored (and even undefined) in the quantum setting and in this work we explores this interplay between privacy, stability and quantum learning. Our definition of stability marks a crucial departure from the classical notion of stability used by [BLM20], in the following sense: a classical learning algorithm is stable if a single function is output by the algorithm with high probability. In contrast we say that a quantum learning algorithm is stable if a collection of quantum states is output with high probability.…”

Section: No-gomentioning

confidence: 99%

“…In order to prove this theorem we borrow the high-level idea from [BLM20] (for the case of Boolean functions). Like [JKT20] which studied the case of online multi-class regression, we borrow the high-level idea from Bun et al [BLM20] (originally developed for the case of Boolean functions) to construct our stable learner: we sample many labelled examples from the distribution D and instead of feeding these examples directly to the black-box RSOA, we plant amongst them some "mistake examples" before giving the processed sample to RSOA. A "mistake example" is an example which is correctly labelled, but on which RSOA would make the wrong prediction.…”

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

“…We proceed similarly but tackle some subtleties related to our learning setting. First, before our work we didn't have an RSOA algorithm which could be used as a black-box in order to emulate the proof-technique of [BLM20] for the case of quantum states. Second, our technique for creating "mistake examples" differs from that of [BLM20].…”

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

“…First, before our work we didn't have an RSOA algorithm which could be used as a black-box in order to emulate the proof-technique of [BLM20] for the case of quantum states. Second, our technique for creating "mistake examples" differs from that of [BLM20]. In the Boolean case, to insert a mistake, it suffices to do the following: suppose c : X → {0, 1} is the unknown target function, they take two candidate set of examples S 1 , S 2 , run two parallel runs of RSOA on S 1 , S 2 , and obtain two output hypothesis functions f 1 , f 2 .…”

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

See 4 more Smart Citations

Private learning implies quantum stability

Arunachalam,

Quek,

Smolin

2021

Preprint

View full text Add to dashboard Cite

Learning an unknown n-qubit quantum state ρ is a fundamental challenge in quantum computing. Information-theoretically, it is well-known that tomography requires exponential in n many copies of an unknown state ρ in order to estimate it up to small trace distance. Motivated by computational learning theory, Aaronson and others introduced several (weaker) learning models: the PAC model of learning quantum states (Proc. of Royal Society A'07), shadow tomography (STOC'18) for learning "shadows" of a quantum state, a learning model that additionally requires learners to be differentially private (STOC'19), and the online model of learning quantum states (NeurIPS'18). In these models it was shown that an unknown quantum state can be learned "approximately well" using linear in n many copies of ρ. But is there any relationship between these learning models? In this paper we prove a sequence of (information-theoretic) implications from differentially-private PAC learning to online learning and then to quantum stability.

show abstract

Section: No-gomentioning

confidence: 66%

Section: No-gomentioning

confidence: 99%

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

Section: Proof Techniques and Further Contributionsmentioning

confidence: 99%

See 3 more Smart Citations

Private learning implies quantum stability

Arunachalam,

Quek,

Smolin

2021

Preprint

View full text Add to dashboard Cite

show abstract

On the Equivalence between Online and Private Learnability beyond Binary Classification

Jung¹,

Kim²,

Tewari³

2020

Preprint

View full text Add to dashboard Cite

Alon et al. [2019] and Bun et al. [2020] recently showed that online learnability and private PAC learnability are equivalent in binary classification. We investigate whether this equivalence extends to multi-class classification and regression. First, we show that private learnability implies online learnability in both settings. Our extension involves studying a novel variant of the Littlestone dimension that depends on a tolerance parameter and on an appropriate generalization of the concept of threshold functions beyond binary classification. Second, we show that while online learnability continues to imply private learnability in multi-class classification, current proof techniques encounter significant hurdles in the regression setting. While the equivalence for regression remains open, we provide non-trivial sufficient conditions for an online learnable class to also be privately learnable.

show abstract

A Limitation of the PAC-Bayes Framework

Livni,

Moran

2020

Preprint

Self Cite

View full text Add to dashboard Cite

PAC-Bayes is a useful framework for deriving generalization bounds which was introduced by McAllester ('98). This framework has the flexibility of deriving distribution-and algorithm-dependent bounds, which are often tighter than VC-related uniform convergence bounds. In this manuscript we present a limitation for the PAC-Bayes framework. We demonstrate an easy learning task which is not amenable to a PAC-Bayes analysis.Specifically, we consider the task of linear classification in 1D; it is well-known that this task is learnable using just O(log(1/δ)/ǫ) examples. On the other hand, we show that this fact can not be proved using a PAC-Bayes analysis: for any algorithm that learns 1-dimensional linear classifiers there exists a (realizable) distribution for which the PAC-Bayes bound is arbitrarily large.

show abstract

An Equivalence Between Private Classification and Online Prediction

Cited by 5 publications

References 34 publications

Private learning implies quantum stability

Private learning implies quantum stability

On the Equivalence between Online and Private Learnability beyond Binary Classification

A Limitation of the PAC-Bayes Framework

Contact Info

Product

Resources

About