Differential Privacy for Growing Databases

Abstract

We study the design of differentially private algorithms for adaptive analysis of dynamically growing databases, where a database accumulates new data entries while the analysis is ongoing. We provide a collection of tools for machine learning and other types of data analysis that guarantee differential privacy and accuracy as the underlying databases grow arbitrarily large. We give both a general technique and a specific algorithm for adaptive analysis of dynamically growing databases. Our general technique is illustrated by two algorithms that schedule black box access to some algorithm that operates on a fixed database to generically transform private and accurate algorithms for static databases into private and accurate algorithms for dynamically growing databases. These results show that almost any private and accurate algorithm can be rerun at appropriate points of data growth with minimal loss of accuracy, even when data growth is unbounded. Our specific algorithm directly adapts the private multiplicative weights algorithm of [HR10] to the dynamic setting, maintaining the accuracy guarantee of the static setting through unbounded data growth. Along the way, we develop extensions of several other differentially private algorithms to the dynamic setting, which may be of independent interest for future work on the design of differentially private algorithms for growing databases.

Continual and Sliding Window Release for Private Empirical Risk Minimization

“…They consider a single bit of input at every round (such as a single element possibly being added to a the set) and publish a differentially private count in every round. A question of adaptive analysis on evolving data was considered in Cummings et al (2018).…”

“…Building on previous works (Blum, Ligett, and Roth 2013;Hardt and Rothblum 2010) on adaptive queries, Cummings et al (2018) describe releasing responses to a broader class of queries. Instead of every round, these results are published at exponentially growing intervals.…”

“…Certain important classes of functions have smaller sensitivity that yields more efficient algorithms. Linear queries over histograms was considered in (Cummings et al 2018;Hardt and Rothblum 2010;Blum, Ligett, and Roth 2013) etc. Suppose U is a finite data universe of size N , and D ∈ U n is a database.…”

“…As a warm-up, we describe a mechanism to release models with a constant -differential privacy guarantee. This approach does not achieve the continual release property of producing an output at each round trained using all previous data, but releases these global outputs at exponentially growing intervals (such as in Cummings et al (2018)).…”

“…In Chan, Shi, and Song (2011); Dwork et al (2010), specific simple statistics such as a count are released at constant intervals of time. The work in Cummings et al (2018) can operate with more general queries such as linear histogram queries and empirical risk minimization (ERM), but publishes models at exponentially growing intervals. In contrast to these approaches, our priority is to maintain upto-date private models with accuracy guarantees for "recent data".…”