A Private and Computationally-Efficient Estimator for Unbounded Gaussians

Kamath, Gautam; Mouzakis, Argyris; Singhal, Vikrant; Steinke, Thomas; Ullman, Jonathan

doi:10.48550/arxiv.2111.04609

Cited by 4 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an independent work, Kamath, Mouzakis, Singhal, Steinke, and Ullman [22] propose a different polynomial time algorithm for privately learning unbounded high-dimensional Gaussian distributions. At a high-level, they devise an iterative preconditioning method for the covariance matrix (in the spirit of [4,21]).…”

Section: Discussion Of a Concurrent Resultsmentioning

confidence: 99%

“…For learning a subspace approximately, they provided an algorithm that was sample-efficient. The latter was also used (with some modifications) in [22] to efficiently learn the covariance matrix of a Gaussian distribution. In this section, we give a computationally-efficient algorithm for learning a subspace exactly with a slightly different sample complexity (albeit with different assumptions).…”

Section: Learning the Subspacementioning

confidence: 99%

See 1 more Smart Citation

Private and polynomial time algorithms for learning Gaussians and beyond

Ashtiani¹,

Liaw²

2021

Preprint

View full text Add to dashboard Cite

We present a fairly general framework for reducing (ε, δ) differentially private (DP) statistical estimation to its non-private counterpart. As the main application of this framework, we give a polynomial time and (ε, δ)-DP algorithm for learning (unrestricted) Gaussian distributions in R d . The sample complexity of our approach for learning the Gaussian up to total variation distance, matching (up to logarithmic factors) the best known informationtheoretic (non-efficient) sample complexity upper bound (Aden-Ali, Ashtiani, Kamath [1]). In an independent work, Kamath, Mouzakis, Singhal, Steinke, and Ullman [22] proved a similar result using a different approach and with O(d 5/2 ) sample complexity dependence on d.As another application of our framework, we provide the first polynomial time (ε, δ)-DP algorithm for robust learning of (unrestricted) Gaussians.

show abstract

Section: Discussion Of a Concurrent Resultsmentioning

confidence: 99%

Section: Learning the Subspacementioning

confidence: 99%

Private and polynomial time algorithms for learning Gaussians and beyond

Ashtiani¹,

Liaw²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Previous work on mean estimation under local differential privacy (Duchi et al 2018) assumes that D = 1 and deploys a Laplace privacy mechanism, which we show is sub-optimal when D is large. Previous work has noted the difficulty of private estimation with unbounded parameter spaces, both in the central model of privacy (Brunel & Avella-Medina 2020, Kamath et al 2021) and the local model (Duchi et al 2013). In the central model, bounds on the unknown mean can be avoided with careful procedures, but in the local model consistent estimation is impossible when D = ∞, even without contamination (see Appendix G, Duchi et al 2013).…”

Section: A Summary Of Our Contributionsmentioning

confidence: 99%

On robustness and local differential privacy

Li¹,

Berrett²,

Chen³

2022

Preprint

View full text Add to dashboard Cite

It is of soaring demand to develop statistical analysis tools that are robust against contamination as well as preserving individual data owners' privacy. In spite of the fact that both topics host a rich body of literature, to the best of our knowledge, we are the first to systematically study the connections between the optimality under Huber's contamination model and the local differential privacy (LDP) constraints.In this paper, we start with a general minimax lower bound result, which disentangles the costs of being robust against Huber's contamination and preserving LDP. We further study three concrete examples: a two-point testing problem, a potentially-diverging mean estimation problem and a nonparametric density estimation problem. For each problem, we demonstrate procedures that are optimal in the presence of both contamination and LDP constraints, comment on the connections with the state-of-the-art methods that are only studied under either contamination or privacy constraints, and unveil the connections between robustness and LDP via partially answering whether LDP procedures are robust and whether robust procedures can be efficiently privatised. Overall, our work showcases a promising prospect of joint study for robustness and local differential privacy.

show abstract

“…We need to find an approximate range of the minibatch of gradients in order to adaptively truncate the gradients and bound the sensitivity. Inspired by a private preconditioning mechanism designed for mean estimation with unknown covariance from [46], we propose to use privately estimated top eigenvalue of the covariance matrix of the gradients. For details on the version of the histogram learner we use in Alg.…”

Section: Differentially Private Principal Component Analysis (Dp-pca)mentioning

confidence: 99%

“…4 in Appendix E.2, we refer to [55,Lemma D.1]. Unlike the private preconditioning of [46] that estimates all eigenvalues and requires n = O(d 3/2 log(1/δ)/ε) samples, we only require the top eigenvalue and hence the next theorem shows that we only need n = O(d log(1/δ)/ε). Theorem 6.1.…”

Section: Differentially Private Principal Component Analysis (Dp-pca)mentioning

confidence: 99%

DP-PCA: Statistically Optimal and Differentially Private PCA

Liu¹,

Kong²,

Jain³

et al. 2022

Preprint

View full text Add to dashboard Cite

We study the canonical statistical task of computing the principal component from n i.i.d. data in d dimensions under (ε, δ)-differential privacy. Although extensively studied in literature, existing solutions fall short on two key aspects: (i) even for Gaussian data, existing private algorithms require the number of samples n to scale super-linearly with d, i.e., n = Ω(d 3/2 ), to obtain non-trivial results while non-private PCA requires only n = O(d), and (ii) existing techniques suffer from a non-vanishing error even when the randomness in each data point is arbitrarily small. We propose DP-PCA, which is a single-pass algorithm that overcomes both limitations. It is based on a private minibatch gradient ascent method that relies on private mean estimation, which adds minimal noise required to ensure privacy by adapting to the variance of a given minibatch of gradients. For sub-Gaussian data, we provide nearly optimal statistical error rates even for n = Õ(d). Furthermore, we provide a lower bound showing that sub-Gaussian style assumption is necessary in obtaining the optimal error rate.

show abstract

A Private and Computationally-Efficient Estimator for Unbounded Gaussians

Cited by 4 publications

References 14 publications

Private and polynomial time algorithms for learning Gaussians and beyond

Private and polynomial time algorithms for learning Gaussians and beyond

On robustness and local differential privacy

DP-PCA: Statistically Optimal and Differentially Private PCA

Contact Info

Product

Resources

About