Membership Inference Attacks and Defenses in Classification Models

Li, Jiacheng; Li, Ninghui; Ribeiro, Bruno

doi:10.1145/3422337.3447836

Cited by 54 publications

(41 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Exploratory attacks are used to extract information from models, and various defenses have been proposed. The dominant attack most related to our work is the membership inference attack, and many defenses [29,32,42] have been proposed. However, most works assume access to the victim's model.…”

Section: Related Workmentioning

confidence: 99%

“…For example, MemGuard [29] is a state-of-the-art defense that adds noise to the model's output to drop the attack model's performance. Other techniques include adding a regularizer to the model's loss function [32] and applying dropout or model stacking techniques [42]. However, such model modifications are not possible in our setting where we assume no access to the model.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Redactor: A Data-centric and Individualized Defense Against Inference Attacks

Heo¹,

Whang²

2022

Preprint

View full text Add to dashboard Cite

Information leakage is becoming a critical problem as various information becomes publicly available by mistake, and machine learning models train on that data to provide services. As a result, one's private information could easily be memorized by such trained models. Unfortunately, deleting information is out of the question as the data is already exposed to the Web or third-party platforms. Moreover, we cannot necessarily control the labeling process and the model trainings by other parties either. In this setting, we study the problem of targeted disinformation where the goal is to lower the accuracy of inference attacks on a specific target (e.g., a person's profile) only using data insertion. While our problem is related to data privacy and defenses against exploratory attacks, our techniques are inspired by targeted data poisoning attacks with some key differences. We show that our problem is best solved by finding the closest points to the target in the input space that will be labeled as a different class. Since we do not control the labeling process, we instead conservatively estimate the labels probabilistically by combining decision boundaries of multiple classifiers using data programming techniques. We also propose techniques for making the disinformation realistic. Our experiments show that a probabilistic decision boundary can be a good proxy for labelers, and that our approach outperforms other targeted poisoning methods when using end-to-end training on real datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Redactor: A Data-centric and Individualized Defense Against Inference Attacks

Heo¹,

Whang²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Li et al [39] proposed a defense based on a new regularizer (using the maximum mean discrepancy [4]) and mixup-training [85]. This method requires labeled reference data to compute the regularization term.…”

Section: Related Workmentioning

confidence: 99%

Knowledge Cross-Distillation for Membership Privacy

Chourasia¹,

Enkhtaivan²,

Ito³

et al. 2021

Preprint

View full text Add to dashboard Cite

A membership inference attack (MIA) poses privacy risks on the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The stateof-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data to protect but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medical and financial, the availability of public data is not obvious. Moreover, a trivial method to generate the public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs using knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable with those of DMP for the benchmark tabular datasets used in MIA researches, Purchase100 and Texas100, and our defense has much better privacy-utility trade-off than those of the existing defenses without using public data for image dataset CIFAR10.

show abstract

“…The privacy risks of DNNs have already been pointed out, where a DNN is prone to memorizing sensitive information of the training dataset [6][7][8][9]. Taking the membership inference attack (MIA) as an example, an adversary can infer whether a given data sample was used to train a DNN, seriously threatening the individual privacy.…”

Section: Introductionmentioning

confidence: 99%

“…In both prediction confidence and sensitivity measurements, neural network pruning makes the distances between the two vertical lines in the pruned model larger than that in the original model, which indicates a larger confidence gap and sensitivity gap between members and non-members due to pruning. members (i.e., training samples) and non-members (i.e., test samples), such as the different prediction confidences [9,10]. Since most neural network pruning approaches rely on reusing training dataset to fine-tune the parameters after pruning the insignificant parameters, the additional training at the pruned neural network inevitably increases its memorization of the training samples.…”

Section: Introductionmentioning

confidence: 99%

Membership Inference Attacks and Defenses in Neural Network Pruning

Yuan¹,

Zhang²

2022

Preprint

View full text Add to dashboard Cite

Neural network pruning has been an essential technique to reduce the computation and memory requirements for using deep neural networks for resource-constrained devices. Most existing research focuses primarily on balancing the sparsity and accuracy of a pruned neural network by strategically removing insignificant parameters and retraining the pruned model. Such efforts on reusing training samples pose serious privacy risks due to increased memorization, which, however, has not been investigated yet.In this paper, we conduct the first analysis of privacy risks in neural network pruning. Specifically, we investigate the impacts of neural network pruning on training data privacy, i.e., membership inference attacks. We first explore the impact of neural network pruning on prediction divergence, where the pruning process disproportionately affects the pruned model's behavior for members and non-members. Meanwhile, the influence of divergence even varies among different classes in a fine-grained manner. Enlighten by such divergence, we proposed a self-attention membership inference attack against the pruned neural networks. Extensive experiments are conducted to rigorously evaluate the privacy impacts of different pruning approaches, sparsity levels, and adversary knowledge. The proposed attack shows the higher attack performance on the pruned models when compared with eight existing membership inference attacks. In addition, we propose a new defense mechanism to protect the pruning process by mitigating the prediction divergence based on KL-divergence distance, whose effectiveness has been experimentally demonstrated to effectively mitigate the privacy risks while maintaining the sparsity and accuracy of the pruned models.

show abstract

Membership Inference Attacks and Defenses in Classification Models

Cited by 54 publications

References 18 publications

Redactor: A Data-centric and Individualized Defense Against Inference Attacks

Redactor: A Data-centric and Individualized Defense Against Inference Attacks

Knowledge Cross-Distillation for Membership Privacy

Membership Inference Attacks and Defenses in Neural Network Pruning

Contact Info

Product

Resources

About