The Johnson-Lindenstrauss Transform Itself Preserves Differential Privacy

Abstract

This paper proves that an "old dog", namely -the classical Johnson-Lindenstrauss transform, "performs new tricks" -it gives a novel way of preserving differential privacy. We show that if we take two databases, D and with a vector of iid normal Gaussians yields two statistically close distributions in the sense of differential privacy. Furthermore, a small, deterministic and public alteration of the input is enough to assert that all singular values of D are large. We apply the Johnson-Lindenstrauss transform to the task of approximating cut-queries: the number of edges crossing a (S,S)-cut in a graph. We show that the JL transform allows us to publish a sanitized graph that preserves edge differential privacy (where two graphs are neighbors if they differ on a single edge) while adding only O(|S|/ǫ) random noise to any given query (w.h.p). Comparing the additive noise of our algorithm to existing algorithms for answering cut-queries in a differentially private manner, we outperform all others on small cuts (|S| = o(n)).We also apply our technique to the task of estimating the variance of a given matrix in any given direction. The JL transform allows us to publish a sanitized covariance matrix that preserves differential privacy w.r.t bounded changes (each row in the matrix can change by at most a norm-1 vector) while adding random noise of magnitude independent of the size of the matrix (w.h.p). In contrast, existing algorithms introduce an error which depends on the matrix dimensions.

On differentially private low rank approximation

“…In order to point out the differences between our work and the recent papers of [10,3], we give the definition of ( , δ)-differential privacy.…”

“…Comparison to [3]. The recent work of [3] gives ( , δ)-differentially private algorithms for two related problems.…”

“…Second, our notion of privacy is the more stringent -differential privacy. The authors of [3] also give an algorithm for answering directional variance queries for a covariance matrix, again under the ( , δ)-differential privacy requirements. As the authors of [3] point out, this estimation procedure is somewhat weaker than low-rank approximation.…”

“…The recent work of [3] gives ( , δ)-differentially private algorithms for two related problems. The first is an algorithm for releasing the Laplacian matrix of a graph that satisifies ( , δ)-differential privacy with respect to edge changes (i.e changing one 0-1 entry of the matrix).…”

Shortest Paths and Distances with Differential Privacy

“…Gupta, Roth and Ullman [GRU12] show how to answer cut queries with O(|V| 1.5 ) error. Blocki et al [BBDS12] improve the error for small cuts. Relatedly, Gupta et al [GLM + 10] show that we can privately release a cut of close to minimal size with error O(log |V|)/ǫ, and that this is optimal.…”