Untitled

Beimel, Amos; Nissim, Kobbi; Stemmer, Uri

doi:10.4086/toc.2016.v012a001

Cited by 19 publications

(11 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For approximate-private learning, Beimel et al [2016] showed that the dependency of the sample complexity in |X| can be significantly reduced. This, however, came at the cost of increasing the dependency in the dimension d. Specifically, the private learner of Beimel et al [2016] has sample complexity Õ d 3 • 8 log * |X| . We mention that a dependency on log * |X| is known to be necessary [Bun et al, 2015, Alon et al, 2019.…”

Section: Existing and New Resultsmentioning

confidence: 99%

“…Intuitively, this guarantees that the outcome of the learner (the identified hypothesis) leaks very little information on any particular point from the training set. Works in this vein include [Kasiviswanathan et al, 2011, Beimel et al, 2014, 2019b, 2016, Bun et al, 2015, Feldman and Xiao, 2015, Bun et al, 2019, Beimel et al, 2019a, Kaplan et al, 2019, 2020a, Alon et al, 2020, Kaplan et al, 2020b, Bun et al, 2020, Alon et al, 2019, and much more.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Sadigurschi,

Stemmer

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We revisit the fundamental problem of learning Axis-Aligned-Rectangles over a finite grid X d ⊆ R d with differential privacy. Existing results show that the sample complexity of this problem is at most min d• log |X| , d 1.5 • (log * |X|) 1.5 . That is, existing constructions either require sample complexity that grows linearly with log |X|, or else it grows super linearly with the dimension d. We present a novel algorithm that reduces the sample complexity to only O d• (log * |X|) 1.5 , attaining a dimensionality optimal dependency without requiring the sample complexity to grow with log |X|. The technique used in order to attain this improvement involves the deletion of "exposed" data-points on the go, in a fashion designed to avoid the cost of the adaptive composition theorems. The core of this technique may be of individual interest, introducing a new method for constructing statistically-efficient private algorithms.

show abstract

Section: Existing and New Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Sadigurschi,

Stemmer

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We next describe properties of an algorithm A RecConcave of Beimel et al [2016]. This algorithm is given a quasi-concave function Q (defined below) and privately finds a point x such that Q(x) is close to its maximum provided that the maximum of Q(x) is large enough (see ( 2)).…”

Section: A Private Algorithm For Optimizing Quasi-concave Functions -...mentioning

confidence: 99%

“…Proposition 2.11 (Properties of Algorithm A RecConcave [Beimel et al, 2016]). Let Q : X * × X → R be a sensitivity-1 function (that is, for every x ∈ X, the function Q(•, x) has sensitivity 1).…”

Section: A Private Algorithm For Optimizing Quasi-concave Functions -...mentioning

confidence: 99%

Private Center Points and Learning of Halfspaces

Beimel¹,

Moran²,

Nissim³

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We present a private learner for halfspaces over an arbitrary finite domain X ⊂ R d with sample complexity poly(d, 2 log * |X| ). The building block for this learner is a differentially private algorithm for locating an approximate center point of m > poly(d, 2 log * |X| ) points -a high dimensional generalization of the median function. Our construction establishes a relationship between these two problems that is reminiscent of the relation between the median and learning one-dimensional thresholds [Bun et al. FOCS '15]. This relationship suggests that the problem of privately locating a center point may have further applications in the design of differentially private algorithms.We also provide a lower bound on the sample complexity for privately finding a point in the convex hull. For approximate differential privacy, we show a lower bound of m = Ω(d + log * |X|), whereas for pure differential privacy m = Ω(d log |X|).

show abstract

“…We stress that the above is a guide, not a theorem, and indeed often requires modification or weakening of the desired property for a given application. For instance, when Step 3 is replaced with a private algorithm, the result is a semi-private learner for general classes, a weakened model allowing the use of a small amount of public unlabeled data [14,2].…”

Section: Introductionmentioning

confidence: 99%

Realizable Learning is All You Need

Hopkins,

Kane,

Lovett

et al. 2024

TheoretiCS

View full text Add to dashboard Cite

The equivalence of realizable and agnostic learnability is a fundamental phenomenon in learning theory. With variants ranging from classical settings like PAC learning and regression to recent trends such as adversarially robust learning, it's surprising that we still lack a unified theory; traditional proofs of the equivalence tend to be disparate, and rely on strong model-specific assumptions like uniform convergence and sample compression. In this work, we give the first model-independent framework explaining the equivalence of realizable and agnostic learnability: a three-line blackbox reduction that simplifies, unifies, and extends our understanding across a wide variety of settings. This includes models with no known characterization of learnability such as learning with arbitrary distributional assumptions and more general loss functions, as well as a host of other popular settings such as robust learning, partial learning, fair learning, and the statistical query model. More generally, we argue that the equivalence of realizable and agnostic learning is actually a special case of a broader phenomenon we call property generalization: any desirable property of a learning algorithm (e.g. noise tolerance, privacy, stability) that can be satisfied over finite hypothesis classes extends (possibly in some variation) to any learnable hypothesis class.

show abstract

Untitled

Cited by 19 publications

References 33 publications

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Private Center Points and Learning of Halfspaces

Realizable Learning is All You Need

Contact Info

Product

Resources

About