Differentially Private Distributed Principal Component Analysis

Distributed Differentially Private Algorithms for Matrix and Tensor Factorization

“…This paper proposes new privacy-preserving algorithms for distributed PCA and OTD and builds upon our earlier work on distributed differentially private eigenvector calculations [17] and centralized differentially private OTD [21]. It improves on our preliminary works on distributed private PCA [17,22] in terms of efficiency and fault-tolerance. Wang and Anandkumar [23] recently proposed an algorithm for differentially private tensor decomposition using a noisy version of the tensor power iteration [3,8].…”

“…Balcan et al [13] proposed a further improved version using fast sparse subspace embedding [19] and randomized SVD [20]. This paper proposes new privacy-preserving algorithms for distributed PCA and OTD and builds upon our earlier work on distributed differentially private eigenvector calculations [17] and centralized differentially private OTD [21]. It improves on our preliminary works on distributed private PCA [17,22] in terms of efficiency and fault-tolerance.…”

“…One of the reasons that capePCA outperforms conv is the smaller noise variance at the aggregator, as described before. Moreover, capePCA outperforms DPdisPCA because DPdisPCA suffers from a larger variance at the aggregator due to computation of the partial square root of s [17]. However, it should be noted here that DPdisPCA offers a much smaller communication overhead than capePCA.…”

“…Performance Gain with Correlated Noise. The distributed differentiallyprivate PCA algorithm of [17] essentially employs the conventional averaging (when each site sends the full s to the aggregator). Therefore, the gain in performance of the proposed capePCA algorithm over the one in [17] is the same as shown in Proposition 1.…”

“…without correlated noise). We designed the experiments according to Imtiaz and Sarwate [17] using three datasets: a synthetic dataset (D = 200, K = 50) generated with zero mean and a pre-determined covariance matrix, the MNIST dataset (D = 784, K = 50) [31] (MNIST) and the Covertype dataset (D = 54, K = 10) [32] (COVTYPE). The MNIST consists of handwritten digits and has a training set We collected the dataset from the UC Irvine KDD archive [32].…”