Faster private release of marginals on small databases

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work, we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar (Proceedings of the 42nd ACM Symposium on Theory of Computing, STOC'10, pp 705-714. ACM Press, New York, 2010). We apply these techniques to the problem of marginal release. Here, a database refers to a collection of the data of n individuals, each characterized by d binary attributes. A k-way marginal query is specified by a subset S of k attributes, together with a |S|-dimensional binary vector β specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to S agrees with β. Information theoretically, the error complexity of marginal queries-how "wrong" do the answers have to be in order to preserve differential privacy-is well understood: the per-query additive error is known to be at least Ω(min{ √ n, d k/2 }) and at mostÕ( √ nd k/2 /4 ). However, no polynomial time algorithm with error complexity Editors-in-Charge: Siu-Wing Cheng and Olivier Devillers Preliminary version in 123 Discrete Comput Geom (2015) 53:650-673 651as low as the information-theoretic upper bound is known for small n. We present a polynomial time algorithm that matches the best known information-theoretic bounds when k = 2; more generally, by reducing to the case k = 2, for any distribution on marginal queries, our algorithm achieves average error at mostÕ( √ nd k/2 /4 ), an improvement over previous work when k is small and when error o(n) is desirable. Using private boosting, we are also able to give nearly matching worst-case error bounds. Our algorithms are based on the geometric techniques of Nikolov et al. (Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC'13, pp 351-360. ACM Press, New York, 2013), wherein a vector of "sufficiently noisy" answers is projected onto a particular convex body. We reduce the projection step, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body K , find a containing convex body L that one can efficiently optimize over, while keeping the Gaussian width of L small. This reduction is achieved by a careful use of the Frank-Wolfe algorithm.

show abstract

“…Plugging this into (6) and taking square roots completes the analysis of the expected MSE. The tail bound follows analogously.…”

Section: Definition 4 For Given D the Convex Body L Consists Of All mentioning

confidence: 99%

Efficient Algorithms for Privately Releasing Marginals via Convex Relaxations

Dwork

Nikolov

Talwar

2015

Discrete Comput Geom

View full text Add to dashboard Cite

show abstract

“…The most closely related works to ours are those of Thaler, Ullman and Vadhan [32], and Chandrasekaran et al [6] and we discuss these in more detail next. Improving on a long line of work [7,22,25], the authors in [32] show that in time d O( √ k log(1/α)) one can construct a private synopsis of a dataset such that any k-way marginal query can be answered from it, with error α · n, as long as n is at least…”

Section: Related Workmentioning

confidence: 71%

“…the best inefficient mechanisms get error αn as long as n is Ω(k √ dα −2 )). The recent work of Chandrasekaran et al [6] presents a different point in the trade-off: they show that one can get error 0.01n for n at least d 0.51 , with running time…”

Section: Related Workmentioning

confidence: 99%

Using Convex Relaxations for Efficiently and Privately Releasing Marginals

Dwork

Nikolov

Talwar

2014

Proceedings of the Thirtieth Annual Symposium on Computational Geometry

View full text Add to dashboard Cite

Differential privacy is a definition giving a strong privacy guarantee even in the presence of auxiliary information. In this work we pursue the application of geometric techniques for achieving differential privacy, a highly promising line of work initiated by Hardt and Talwar [26], focusing on the problem of marginal release. Here, a database is a collection of the data of n individuals, each characterized by d binary attributes. A k-way marginal query is specified by a subset S of k attributes, together with a |S|-dimensional binary vector β specifying their values. The true answer to this query is a count of the number of people in the database whose attribute vector restricted to S agrees with β.Information theoretically, the error complexity of marginal queries -how "wrong" do the answers have to be in order to preserve differential privacy -is well-understood: the perquery additive error is known to be at least Ω(min{ √ n, d k/2 }) and at mostÕ( √ nd 1/4 , d k/2 ). However, no polynomial time algorithm with error complexity as low as the information theoretic upper bound is known for small n.We present a polynomial time algorithm that matches the best known information-theoretic bounds when k = 2; more generally, by reducing to the case k = 2, for any distribution on marginal queries, our algorithm achieves average error at mostÕ( √ nd k/2 4 ), an improvement over previous work on when k is small and when error o(n) is desirable. Using private boosting we are also able to give nearly matching worst-case error bounds.Our algorithms are based on the geometric techniques of Nikolov, Talwar, and Zhang [29], wherein a vector of "sufficiently noisy" answers is projected onto a particular convex body. We reduce projection, which is expensive, to a simple geometric question: given (a succinct representation of) a convex body K, find a containing convex body L that one can efficiently optimize over, while keeping the Gaussian width of L small. This reduction is achieved by a careful use of the Frank-Wolfe algorithm.

show abstract

“…In fact, when the synthetic data restriction is lifted, there are algorithms (e.g. [HRS12,TUV12,CTUW14,DNT14]) that answer queries from certain exponentially large families in subexponential time. One can view the problem of synthetic data generation as analogous to proper learning.…”

Section: Related Workmentioning

confidence: 99%

Order-Revealing Encryption and the Hardness of Private Learning

Bun

Zhandry

2015

Theory of Cryptography

View full text Add to dashboard Cite

An order-revealing encryption scheme gives a public procedure by which two ciphertexts can be compared to reveal the ordering of their underlying plaintexts. We show how to use order-revealing encryption to separate computationally efficient PAC learning from efficient (ε, δ)-differentially private PAC learning. That is, we construct a concept class that is efficiently PAC learnable, but for which every efficient learner fails to be differentially private. This answers a question of Kasiviswanathan et al. (FOCS '08, SIAM J. Comput. '11).To prove our result, we give a generic transformation from an order-revealing encryption scheme into one with strongly correct comparison, which enables the consistent comparison of ciphertexts that are not obtained as the valid encryption of any message. We believe this construction may be of independent interest.

show abstract

Faster private release of marginals on small databases

Cited by 38 publications

References 43 publications

Efficient Algorithms for Privately Releasing Marginals via Convex Relaxations

Efficient Algorithms for Privately Releasing Marginals via Convex Relaxations

Using Convex Relaxations for Efficiently and Privately Releasing Marginals

Order-Revealing Encryption and the Hardness of Private Learning

Contact Info

Product

Resources

About