Local Differential Privacy for Evolving Data

Abstract

There are now several large scale deployments of differential privacy used to collect statistical information about users. However, these deployments periodically recollect the data and recompute the statistics using algorithms designed for a single use. As a result, these systems do not provide meaningful privacy guarantees over long time scales. Moreover, existing techniques to mitigate this effect do not apply in the "local model" of differential privacy that these systems use. In this paper, we introduce a new technique for local differential privacy that makes it possible to maintain up-to-date statistics over time, with privacy guarantees that degrade only in the number of changes in the underlying distribution rather than the number of collection periods. We use our technique for tracking a changing statistic in the setting where users are partitioned into an unknown collection of groups, and at every time period each user draws a single bit from a common (but changing) group-specific distribution. We also provide an application to frequency and heavy-hitter estimation. IntroductionAfter over a decade of research, differential privacy [12] is moving from theory to practice, with notable deployments by Google [15,6], Apple [2], Microsoft [10], and the U.S. Census Bureau [1]. These deployments have revealed gaps between existing theory and the needs of practitioners. For example, the bulk of the differential privacy literature has focused on the central model, in which user data is collected by a trusted aggregator who performs and publishes the results of a differentially private computation [11]. However, Google, Apple, and Microsoft have instead chosen to operate in the local model [15,6,2,10], where users individually randomize their data on their own devices and send it to a potentially untrusted aggregator for analysis [18]. In addition, the academic literature has largely focused on algorithms for performing one-time computations, like estimating many statistical quantities [7, 22,16] or training a classifier [18,9,4]. Industrial applications, however have focused on tracking statistics about a user population, like the set of most frequently used emojis or words [2]. These statistics evolve over time and so must be re-computed periodically.Together, the two problems of periodically recomputing a population statistic and operating in the local model pose a challenge. Naïvely repeating a differentially private computation causes the privacy loss to degrade as the square root of the number of recomputations, quickly leading to enormous values of ǫ. This naïve strategy is what is used in practice [15,6,2]. As a result, Tang et al. [23] discovered that the privacy parameters guaranteed by Apple's implementation of differentially private data 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada.

Locally Differentially Private Frequency Estimation with Consistency

“…LDP frequency oracle is also a building block for other analytical tasks, e.g., finding heavy hitters [4], [7], [34], frequent itemset mining [26], [33], releasing marginals under LDP [27], [8], [38], key-value pair estimation [37], [15], evolving data monitoring [18], [13], and (multi-dimensional) range analytics [32], [22]. Mean estimation is also a building block in LDP; most of existing work transforms the numerical value to a discrete value using stochastic round, and then apply frequency oracles [11], [29], [24].…”

The Role of Interactivity in Local Differential Privacy

“…Not every protocol is 1-compositional: exceptions include RAPPOR[20] and the evolving data protocol of Joseph et al[24].…”

Local differential privacy for human-centered computing

“…The Hadamard response [23] uses the Hadamard transform instead of the hash function; this is because the actual calculation of Hadamard entries is easier, the server-side aggregates report faster. Joseph et al [24] propose a technique that repeatedly recomputes a statistic with the error which leads to the decays of errors; it happens when the statistic changes significantly rather than the current value of the statistic is recomputed. Wang et al [25] introduce a method that adds postprocessing steps to frequency estimations to make them consistent while achieving high accuracy for a wide range of tasks.…”