Algorithmic Stability for Adaptive Data Analysis

Bassily, Raef; Nissim, Kobbi; Smith, Adam; Steinke, Thomas; Stemmer, Uri; Ullman, Jonathan

doi:10.1137/16m1103646

Cited by 57 publications

(100 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we wish to make a connection to a remarkable property of differential privacy: it protects against false discoveries due to adaptive data analysis, where an analysis is informed by prior interactions with the same database [22,21,3]. Adaptivity is ubiquitous in practice as the analyst is often not clear a priori what are the right questions to ask about a database.…”

Section: Discussionmentioning

confidence: 99%

Differentially private false discovery rate control

Dwork

Zhang

2021

JPC

View full text Add to dashboard Cite

Differential privacy provides a rigorous framework for privacy-preserving data analysis. This paper proposes the first differentially private procedure for controlling the false discovery rate (FDR) in multiple hypothesis testing. Inspired by the Benjamini-Hochberg procedure (BHq), our approach is to first repeatedly add noise to the logarithms of the p-values to ensure differential privacy and to select an approximately smallest p-value serving as a promising candidate at each iteration; the selected p-values are further supplied to the BHq and our private procedure releases only the rejected ones. Moreover, we develop a new technique that is based on a backward submartingale for proving FDR control of a broad class of multiple testing procedures, including our private procedure, and both the BHq step- up and step-down procedures. As a novel aspect, the proof works for arbitrary dependence between the true null and false null test statistics, while FDR control is maintained up to a small multiplicative factor.

show abstract

Section: Discussionmentioning

confidence: 99%

Differentially private false discovery rate control

Dwork

Zhang

2021

JPC

View full text Add to dashboard Cite

show abstract

“…The application that is most similar to our work is Hassidim et al [2020], which uses differential privacy of the internal randomness of an algorithm (as we do) to reduce streaming algorithms with guarantees against adaptive adversarial streams to streaming algorithms with guarantees against oblivious adversaries. Our techniques differ; while Hassidim et al [2020] reduce to the so-called "transfer theorem for linear and low sensitivity queries" developed over a series of works Dwork et al [2015c], Bassily et al [2021], Jung et al [2020], we use a more general connection between differential privacy and "max-information" established in Dwork et al [2015b], Rogers et al [2016].…”

Section: Related Workmentioning

confidence: 99%

Adaptive Machine Unlearning

Gupta,

Jung,

Neel

et al. 2021

Preprint

View full text Add to dashboard Cite

Data deletion algorithms aim to remove the influence of deleted data points from trained models at a cheaper computational cost than fully retraining those models. However, for sequences of deletions, most prior work in the non-convex setting gives valid guarantees only for sequences that are chosen independently of the models that are published. If people choose to delete their data as a function of the published models (because they don't like what the models reveal about them, for example), then the update sequence is adaptive. In this paper, we give a general reduction from deletion guarantees against adaptive sequences to deletion guarantees against non-adaptive sequences, using differential privacy and its connection to max information. Combined with ideas from prior work which give guarantees for non-adaptive deletion sequences, this leads to extremely flexible algorithms able to handle arbitrary model classes and training methodologies, giving strong provable deletion guarantees for adaptive deletion sequences. We show in theory how prior work for non-convex models fails against adaptive deletion sequences, and use this intuition to design a practical attack against the SISA algorithm of Bourtoule et al. [2021] on CIFAR-10, MNIST, Fashion-MNIST.If the request is for rectification or erasure of the data, this may not be possible without re-training the model...or deleting the model altogether [ICO, 2020]. Fully retraining the model every time a deletion request is received can be prohibitive in terms of both time and money-especially for large models and frequent deletion requests. The problem of data deletion (also known as machine unlearning) is to find an algorithmic middle ground between the compliant but impractical baseline of retraining, and the potentially illegal standard of doing nothing. We iteratively update

show abstract

“…In addition to the above privacy analysis, our characterization also allows us to obtain simple utility bounds (these are known [27] and also hold for the exponential mechanism [28,2], but we provide a simple self-contained proof): Proposition 11. Let D be a dataset and let q be a quality function with sensitivity ∆ and set of outcomes…”

Section: Proposition 6 Report Noisy Max With Exponential Noise (Algor...mentioning

confidence: 99%

“…where the probability is only over the randomness of M . 2 Intuitively, differential privacy guarantees that the output distribution of M does not depend too much on any individual's data. Thus a potential adversary cannot infer much about any individual from the output.…”

Section: Introductionmentioning

confidence: 99%

The Permute-and-Flip Mechanism is Identical to Report-Noisy-Max with Exponential Noise

Ding,

Kifer,

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The permute-and-flip mechanism is a recently proposed differentially private selection algorithm that was shown to outperform the exponential mechanism. In this paper, we show that permute-and-flip is equivalent to the well-known report noisy max algorithm with exponential noise. This equivalence is used to simplify the utility and privacy analysis, and extend it to bounded range and concentrated differential privacy.

show abstract

Algorithmic Stability for Adaptive Data Analysis

Cited by 57 publications

References 8 publications

Differentially private false discovery rate control

Differentially private false discovery rate control

Adaptive Machine Unlearning

The Permute-and-Flip Mechanism is Identical to Report-Noisy-Max with Exponential Noise

Contact Info

Product

Resources

About