One-shot Empirical Privacy Estimation for Federated Learning

Abstract

Privacy auditing techniques for differentially private (DP) algorithms are useful for estimating the privacy loss to compare against analytical bounds, or empirically measure privacy in settings where known analytical bounds on the DP loss are not tight. However, existing privacy auditing techniques usually make strong assumptions on the adversary (e.g., knowledge of intermediate model iterates or the training data distribution), are tailored to specific tasks and model architectures, and require retraining the model many times (typically on the order of thousands). These shortcomings make deploying such techniques at scale difficult in practice, especially in federated settings where model training can take days or weeks. In this work, we present a novel "one-shot" approach that can systematically address these challenges, allowing efficient auditing or estimation of the privacy loss of a model during the same, single training run used to fit model parameters. Our privacy auditing method for federated learning does not require a priori knowledge about the model architecture or task. We show that our method provides provably correct estimates for privacy loss under the Gaussian mechanism, and we demonstrate its performance on a well-established FL benchmark dataset under several adversarial models.

On the Risks of Collecting Multidimensional Data Under Local Differential Privacy

“…Our LDP-Auditor framework is described in Algorithm 1, which is a slightly modification of [19]. The main difference between DP-SGD auditing [2,19,23,[29][30][31][32][33] to LDP auditing is the test statistic, i.e., distinguishability of input values in LDP versus distinguishability of neighboring datasets in central DP. From Equation (1), our goal is to estimate the two quantities p0 = Pr[M (𝑣 1 ) = 𝑦] and p1 = Pr[M (𝑣 2 ) = 𝑦] to compute 𝜖 𝑙𝑏 = ln ( p0 / p1 ).…”

“…( 3) With probability 𝑝 (as indicated in Equation ( 3)), the index at position 𝑣 (representing the user's true value) is flipped to 1. (4) Missing step: if v 𝑣 was set to 1 in step (2) but not in step (3), there should be a correction to revert it back to 0. This is a simple mistake that can be easily corrected and we have contacted the authors to address it accordingly.…”

“…On the one hand, an (L)DP mechanism is accompanied by a mathematical proof that establishes a theoretical upper bound for the privacy loss through the privacy parameter 𝜖. On the other hand, the recent and emerging field of DP auditing (e.g., [2,19,23,25,[29][30][31][32][33]) estimates an empirical lower bound for the privacy loss 𝜖 through privacy attacks. For instance, DP auditing has been used for evaluating the mathematical analysis for the widely used differentially private stochastic gradient descent (DP-SGD) [1] algorithm in the domain of privacy-preserving machine learning.…”

The Role of Dialogue in the Teaching of Reading in South Africa

“…Improvements have also been made to the basic statistical techniques for estimating the ε parameter, for example by using Log-Katz confidence intervals [LMFLZWRFT22], Bayesian techniques [ZBWT-SRPNK22], or auditing algorithms in different privacy definitions [NHSBTJCT23]. Andrew, Kairouz, Oh, Oprea, McMahan, and Suriyakumar [AKOOMS23] build on the observation that, when performing membership inference, analyzing the case where the data is not included does not require re-running the algorithm; instead we can re-sample the excluded data point; if the data points are i.i.d. from a nice distribution, this permits closed-form analysis of the excluded case.…”