Differentially Private K-Means Clustering and a Hybrid Approach to Private Optimization

Abstract

k -means clustering is a widely used clustering analysis technique in machine learning. In this article, we study the problem of differentially private k -means clustering. Several state-of-the-art methods follow the single-workload approach, which adapts an existing machine-learning algorithm by making each step private. However, most of them do not have satisfactory empirical performance. In this work, we develop techniques to analyze the empirical error behaviors of one of the state-of-the-art single-workload approaches, DPLloyd, which is a differentially private version of the Lloyd algorithm for k >-means clustering. Based on the analysis, we propose an improvement of DPLloyd. We also propose a new algorithm for k -means clustering from the perspective of the noninteractive approach, which publishes a synopsis of the input dataset and then runs k -means on synthetic data generated from the synopsis. We denote this approach by EUGkM. After analyzing the empirical error behaviors of EUGkM, we further propose a hybrid approach that combines our DPLloyd improvement and EUGkM. Results from extensive and systematic experiments support our analysis and demonstrate the effectiveness of the DPLloyd improvement, EUGkM, and the hybrid approach.

Differential identifiability clustering algorithms for big data analysis

“…However, we expect differential identifiability can be applied to more complex privacy preservation issues to evaluate performance, such as data mining in big data. Table 2 [13,[33][34][35][36][37] compares some privacy-preserving k-means algorithms. Currently, differential identifiability can provide an easy parameterization approach.…”

Privacy‐preserving constrained spectral clustering algorithm for large‐scale data sets

“…Su et al [35] comprehensively evaluated the above-mentioned differentially private k-means algorithms, and provided an improved version of McSherry's approach [31] and a noninteractive method for differentially private k-means clustering named EUGkM. Subsequently, they [11] further put forward a novel hybrid method that combines EUGkM with the approach in [31]. Holohan et al [36] presented the IBM Differential Privacy Library, an open source library for differential privacy applications.…”

“…So far, many traditional data mining algorithms have been extended to satisfy differential privacy, e.g. k-means [11], k-nearest neighbour classification [12], random forest [13], frequent itemset mining [14,15] and so on. Nevertheless, when focusing on spectral clustering, there are few works with regard to differentially private spectral clustering algorithms.…”

User Behavior Clustering Scheme With Automatic Tagging Over Encrypted Data

“…Class features while ensuring differential privacy. Su et al [23] also applies differential privacy to the privacy-protected K-means algorithm. However, too many cluster iterations mean less privacy budget for each iteration.…”