Gaussian Mixture Models for Classification and Hypothesis Tests Under Differential Privacy

Abstract

Many statistical models are constructed using very basic statistics: mean vectors, variances, and covariances. Gaussian mixture models are such models. When a data set contains sensitive information and cannot be directly released to users, such models can be easily constructed based on noise added query responses. The models nonetheless provide preliminary results to users. Although the queried basic statistics meet the differential privacy guarantee, the complex models constructed using these statistics may not meet the differential privacy guarantee. However it is up to the users to decide how to query a database and how to further utilize the queried results. In this article, our goal is to understand the impact of differential privacy mechanism on Gaussian mixture models. Our approach involves querying basic statistics from a database under differential privacy protection, and using the noise added responses to build classifier and perform hypothesis tests. We discover that adding Laplace noises may have a non-negligible effect on model outputs. For example variance-covariance matrix after noise addition is no longer positive definite. We propose a heuristic algorithm to repair the noise added variance-covariance matrix. We then examine the classification error using the noise added responses, through experiments with both simulated data and real life data, and demonstrate under which conditions the impact of the added noises can be reduced. We compute the exact type I and type II errors under differential privacy for one sample z test, one sample t test, and two sample t test with equal variances. We then show under which condition a hypothesis test returns reliable result given differentially private means, variances and covariances.

A Comprehensive Survey on Local Differential Privacy toward Data Statistics and Analysis

“…Many existing studies looked at the intersection of DP and hypotheses testing [ 178 , 179 , 180 , 181 ]. The private hypothesis testing under LDP has also been studied in [ 115 , 182 , 183 , 184 , 185 , 186 ], including identity and independence testing, Z-test, and distribution testing.…”

A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysis

“…Many existing studies have looked at the intersection of DP and hypotheses testing [179]- [182]. The private hypothesis testing under LDP has also been studied in [115], [183]- [187], including identity and independence testing, Z-test, and distribution testing.…”

Differentially Private Outlier Detection in Correlated Data

“…In addition, differentially private outlier detection using Monte Carlo (MC) approaches were proposed in [20], [21], as well as using machine learning-based techniques in [22]. Furthermore, differentially private statistical tests were studied for the data of individual agents under a Gaussian assumption in [23] and [24], in order to decide whether or not the mean of a sequence of independent and identically distributed (i.i.d.) scalar Gaussian random variables differed from a given value.…”