Differentially private sub-Gaussian location estimators

Abstract

We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differential privacy and show that a naive application of the Laplace mechanism can lead to sub-optimal results. We design two private algorithms for estimating the median that lead to estimators with sub-Gaussian type errors. Unlike most existing differentially private median estimators, both algorithms are well defined for unbounded random variables that are not even required to have finite moments. We then turn to the problem of sub-Gaussian mean estimation and show that under heavy tails natural differentially private alternatives lead to strictly worse deviations than their non-private sub-Gaussian counterparts. This is in sharp contrast with recent results that show that from an asymptotic perspective the cost of differential privacy is negligible.

Privacy-Preserving Parametric Inference: A Case for Robust Statistics

“…Indeed, it might not be checkable by the users and one would like to have such a guarantee to hold over all possible configurations of the data. One possible way of tackling this problem is to let the algorithm halt with an output "No Reply" when this assumption fails (Dwork and Lei, 2009;Avella-Medina and Brunel, 2019). Condition 3 is a smoothness condition on Ψ at F n , similar to Condition 4 in Chaudhuri and Hsu (2012).…”

Concentration of the exponential mechanism and differentially private multivariate medians

“…Recall that estimating a parameter from an unbounded parameter space, i.e., unbounded estimation, is non-trivial in the setting of differential privacy, e.g., see [3]. Corollary 3.1 shows that choosing σ p relatively large allows one to perform unbounded estimation at a cost of only log d to the sample complexity.…”

“…On the other hand, considerably less attention has been given to robust differentially private location estimation via a median. To date, the literature has focused on univariate median estimation: Dwork and Lei [19] introduced an approximately differentially private median estimator with asymptotic consistency guarantees, Avella-Medina and Brunel [3], Brunel and Avella-Medina [9] then introduced several median estimators which achieve sub-Gaussian error rates, and Tzamos et al [55] introduce median estimators with optimal sample complexity. Private estimation of multivariate medians has not been addressed.…”