A compressive multi-kernel method for privacy-preserving machine learning

Abstract

As the analytic tools become more powerful, and more data are generated on a daily basis, the issue of data privacy arises. This leads to the study of the design of privacypreserving machine learning algorithms. Given two objectives, namely, utility maximization and privacy-loss minimization, this work is based on two previously non-intersecting regimes -Compressive Privacy and multi-kernel method. Compressive Privacy is a privacy framework that employs utility-preserving lossy-encoding scheme to protect the privacy of the data, while multi-kernel method is a kernel-based machine learning regime that explores the idea of using multiple kernels for building better predictors. In relation to the neural-network architecture, multi-kernel method can be described as a two-hidden-layered network with its width proportional to the number of kernels.The compressive multi-kernel method proposed consists of two stages -the compression stage and the multi-kernel stage. The compression stage follows the Compressive Privacy paradigm to provide the desired privacy protection. Each kernel matrix is compressed with a lossy projection matrix derived from the Discriminant Component Analysis (DCA). The multikernel stage uses the signal-to-noise ratio (SNR) score of each kernel to non-uniformly combine multiple compressive kernels.The proposed method is evaluated on two mobile-sensing datasets -MHEALTH and HAR -where activity recognition is defined as utility and person identification is defined as privacy. The results show that the compression regime is successful in privacy preservation as the privacy classification accuracies are almost at the random-guess level in all experiments. On the other hand, the novel SNR-based multi-kernel shows utility classification accuracy improvement upon the state-of-the-art in both datasets. These results indicate a promising direction for research in privacy-preserving machine learning.

Sensor-based human activity recognition using fuzzified deep CNN architecture with λmax method

“…Initially, adapted Deep Boltzmann Machines (Li et al, 2014) for position-based HAR feature extraction. Various standard machine learning algorithms (Cer on et al, 2018;Liang et al, 2018;Coskun et al, 2015;Chanyaswad et al, 2017), including a decision tree, k-nearest neighbor (kNN), support vector machine (SVM) and ensemble approaches, including random forest, boosting, and bagging were experimented on UCI data. With an integrated multiple Hidden Markov Models (HMM) classifier with mixture-of-templates and kNN ensemble (Kim et al, 2016) have achieved 92.6% accuracy.…”

Data-driven Regularized Inference Privacy

“…The paper [9] introduced a dimension reduction subspace approach. As an extension, the paper [25] studied the design of privacy-preserving mechanisms based on compressive privacy and multi-kernel methods. The reference [26] proposed a nonparametric learning approach to design privacy mappings to distort sensors' observations.…”

“…In a similar setting, the paper [27] proposed a multilayer nonlinear processing procedure to distort sensors' data. However, these works do not cater for a wide range of privacy problems since [9] focuses on linear utility and privacy spaces and [25]- [27] target at classification problems only. More closely related works are [28]- [30], which use auto-encoders to learn representations that are invariant to certain sensitive factors in the data while retaining as much of the remaining information as possible.…”

Privacy preservation for linear and learning based inference systems

“…a privacy space. This work was extended by [97] through a combination of compressive privacy and multi-kernel method. The works [45,98] enabled the detection of a public hypothesis while protecting a private hypothesis by learning mechanisms to distort sensors' measurements.…”

“…(1) The methods in [60,97] are restricted to deal with linear utility and privacy subspaces and hence are incapable of managing nonlinear relationships; (2) Choosing a standard isotropic Gaussian prior and a Gaussian posterior for encoded data and decoded data in [99,100] is too simple to model complex data. Inspired by the generative adversarial networks [102], the state-of-the-art generative adversarial privacy (GAP) method proposed by [103105] formulated the utility-privacy tradeo as a competing game between a privatizer and an adversary.…”