No free lunch in data privacy

Abstract

Differential privacy is a powerful tool for providing privacypreserving noisy query answers over statistical databases. It guarantees that the distribution of noisy query answers changes very little with the addition or deletion of any tuple. It is frequently accompanied by popularized claims that it provides privacy without any assumptions about the data and that it protects against attackers who know all but one record.In this paper we critically analyze the privacy protections offered by differential privacy.First, we use a no-free-lunch theorem, which defines nonprivacy as a game, to argue that it is not possible to provide privacy and utility without making assumptions about how the data are generated. Then we explain where assumptions are needed. We argue that privacy of an individual is preserved when it is possible to limit the inference of an attacker about the participation of the individual in the data generating process. This is different from limiting the inference about the presence of a tuple (for example, Bob's participation in a social network may cause edges to form between pairs of his friends, so that it affects more than just the tuple labeled as "Bob"). The definition of evidence of participation, in turn, depends on how the data are generated -this is how assumptions enter the picture. We explain these ideas using examples from social network research as well as tabular data for which deterministic statistics have been previously released. In both cases the notion of participation varies, the use of differential privacy can lead to privacy breaches, and differential privacy does not always adequately limit inference about participation.

Calibrating Noise to Sensitivity in Private Data Analysis

“…The definition of semantic privacy given in this paper provides guarantees on how a Bayesian adversary's posterior distribution compares to its prior, under assumptions on the form of the adversary's prior. Such assumptions are known to be necessary (Dwork, 2006;Dwork and Naor, 2010;Kifer and Machanavajjhala, 2011). In contrast, one may also formulate definitions by comparing the adversary's posterior distributions in different settings (say, assuming that someone's data was or was not used in the computation) ; Bassily et al (2013).…”

Differential Privacy with Bounded Priors

“…It is well known that data privacy against arbitrary priors cannot be guaranteed if some reasonable level of utility is to be achieved. This fact, known as the no-free-lunch-theorem, was first introduced by Kifer and Machanavajjhala [11], and reformulated by Li et al [14] as part of their framework. We now give the formal definition of γ-positive membership-privacy under a family of prior distributions D, which we denote as (γ, D)-PMP.…”

Differential Privacy in Practice

“…Kifer and Machanavajjhala [61] critically analysed the privacy protections under differential privacy via nonprivacy games. They addressed several popular misconceptions about differential privacy, including: that it makes no assumptions about how data are generated; that it protects an individual's information even if an attacker knows about all other individuals in the data; and that it is robust to arbitrary background knowledge.…”