Private PAC learning implies finite Littlestone dimension

Alon, Noga; Livni, Roi; Malliaris, M.; Moran, Shay

doi:10.1145/3313276.3316312

Cited by 29 publications

(23 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The following well‐known result, which appears in , relates the degree of stability of a graph to its tree bound (see also [, Appendix B] for another, recent exposition on the proof). Theorem For each integer

k

there exists

d = d false(k false) < 2^{k + 2} - 2

such that if

normalΓ

is a

k

‐stable graph, then its tree bound

d (Γ)

is at most

d

.…”

Section: Building a Tree In The Absence Of Efficient Regularitymentioning

confidence: 99%

See 1 more Smart Citation

Stable arithmetic regularity in the finite field model

Terry

Wolf

2018

Bull. London Math. Soc.

View full text Add to dashboard Cite

The arithmetic regularity lemma for Fpn, proved by Green in 2005, states that given a subset A⊆double-struckFpn, there exists a subspace H⩽double-struckFpn of bounded codimension such that A is Fourier‐uniform with respect to almost all cosets of H. It is known that in general, the growth of the codimension of H is required to be of tower type depending on the degree of uniformity, and that one must allow for a small number of non‐uniform cosets. Our main result is that, under a natural model‐theoretic assumption of stability, the tower‐type bound and non‐uniform cosets in the arithmetic regularity lemma are not necessary. Specifically, we say that a set A⊆double-struckFpn is k‐stable if there are no a1,…,ak,b1,…,bk∈double-struckFpn such that ai+bj∈A if and only if i⩽j. We prove an arithmetic regularity lemma for k‐stable subsets A⊆double-struckFpn in which the bound on the codimension of the subspace is a polynomial (depending on k) in the degree of uniformity, and in which there are no non‐uniform cosets. This result is an arithmetic analogue of the stable graph regularity lemma proved by Malliaris and Shelah.

show abstract

k

there exists

d = d false(k false) < 2^{k + 2} - 2

such that if

normalΓ

is a

k

‐stable graph, then its tree bound

d (Γ)

is at most

d

.…”

Section: Building a Tree In The Absence Of Efficient Regularitymentioning

confidence: 99%

“…The following well-known result, which appears in [17], relates the degree of stability of a graph to its tree bound (see also [2,Appendix B] for another, recent exposition on the proof).…”

Section: Building a Tree In The Absence Of Efficient Regularitymentioning

confidence: 99%

Stable arithmetic regularity in the finite field model

Terry

Wolf

2018

Bull. London Math. Soc.

View full text Add to dashboard Cite

show abstract

“…For approximate differential privacy, the current understanding is more limited. Recent results established that the class of one-dimensional thresholds over a domain X ⊆ R requires sample complexity between Ω(log * |X|) and 2 O(log * |X|) (Beimel et al [2013b], Bun et al [2015], Bun [2016], Alon et al [2018]). On the one hand, these results establish a separation between what can be learned with or without privacy, as they imply that privately learning one-dimensional thresholds over an infinite domain is impossible.…”

Section: Introductionmentioning

confidence: 99%

Private Center Points and Learning of Halfspaces

Beimel¹,

Moran²,

Nissim³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We present a private learner for halfspaces over an arbitrary finite domain X ⊂ R d with sample complexity poly(d, 2 log * |X| ). The building block for this learner is a differentially private algorithm for locating an approximate center point of m > poly(d, 2 log * |X| ) points -a high dimensional generalization of the median function. Our construction establishes a relationship between these two problems that is reminiscent of the relation between the median and learning one-dimensional thresholds [Bun et al. FOCS '15]. This relationship suggests that the problem of privately locating a center point may have further applications in the design of differentially private algorithms.We also provide a lower bound on the sample complexity for privately finding a point in the convex hull. For approximate differential privacy, we show a lower bound of m = Ω(d + log * |X|), whereas for pure differential privacy m = Ω(d log |X|).

show abstract

“…(A threshold function is a binary function that evaluates to 1 on some prefix of the domain. 1 ) The goal is to generalize the training data into a hypothesis h that predicts the labels of unseen examples. In this paper we study this problem under the constraint of differential privacy.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, Bun et al [5] showed a lower bound of Ω(log * |X|) on the sample complexity of every approximateprivate learner for thresholds that outputs a hypothesis that is itself a threshold function (such a learner is called proper). Recently, Alon et al [1] showed that a lower bound of Ω(log * |X|) holds even for improper learners, i.e., for learners whose output hypothesis is not restricted to being a threshold function. To summarize, our current understanding of the task of privately learning thresholds places its sample complexity somewhere between Ω(log * |X|) and Õ 2 log * |X| , a gap which is exponential in log * |X|, where at least three different algorithms are known with sample complexity 2 O(log * |X|) .…”

Section: Introductionmentioning

confidence: 99%

Privately Learning Thresholds: Closing the Exponential Gap

Kaplan,

Ligett,

Mansour

et al. 2019

Preprint

View full text Add to dashboard Cite

We study the sample complexity of learning threshold functions under the constraint of differential privacy. It is assumed that each labeled example in the training data is the information of one individual and we would like to come up with a generalizing hypothesis h while guaranteeing differential privacy for the individuals. Intuitively, this means that any single labeled example in the training data should not have a significant effect on the choice of the hypothesis. This problem has received much attention recently; unlike the non-private case, where the sample complexity is independent of the domain size and just depends on the desired accuracy and confidence, for private learning the sample complexity must depend on the domain size X (even for approximate differential privacy). Alon et al. (STOC 2019) showed a lower bound of Ω(log * |X|) on the sample complexity and Bun et al. (FOCS 2015) presented an approximate-private learner with sample complexity Õ 2 log * |X| . In this work we reduce this gap significantly, almost settling the sample complexity. We first present a new upper bound (algorithm) of Õ (log * |X|) 2 on the sample complexity and then present an improved version with sample complexity Õ (log * |X|) 1.5 .

show abstract

Private PAC learning implies finite Littlestone dimension

Cited by 29 publications

References 29 publications

Stable arithmetic regularity in the finite field model

Stable arithmetic regularity in the finite field model

Private Center Points and Learning of Halfspaces

Privately Learning Thresholds: Closing the Exponential Gap

Contact Info

Product

Resources

About