On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models

Zhao, Benjamin Zi Hao; Agrawal, Aviral; Coburn, Catisha; Asghar, Hassan Jameel; Bhaskar, Raghav; Kâafar, Mohamed Ali; Webb, Darren; Dickinson, Peter

doi:10.1109/eurosp51992.2021.00025

Cited by 17 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To generate high utility data whose distribution is similar to the original data distribution, these DP methods guarantee DP with large values of

. As described in [ 7 ], large values of

cause to expose sensitive information under privacy attacks targeting data. On the other hand, while generating data with high privacy guarantees, these DP methods decrease the utility of data.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, another factor of concern is that these anonymization techniques should not only protect data where sensitive information is identified from the dataset, but also data where sensitive information is identifiablefrom the correlation of multiple datasets. However, many studies showed that ML models are vulnerable to an increasing array of various privacy attacks willing to expose an individual [ 6 , 7 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Task-Specific Adaptive Differential Privacy Method for Structured Data

Utaliyeva

Shin

Choi

2023

Sensors

View full text Add to dashboard Cite

Data are needed to train machine learning (ML) algorithms, and in many cases often include private datasets that contain sensitive information. To preserve the privacy of data used while training ML algorithms, computer scientists have widely deployed anonymization techniques. These anonymization techniques have been widely used but are not foolproof. Many studies showed that ML models using anonymization techniques are vulnerable to various privacy attacks willing to expose sensitive information. As a privacy-preserving machine learning (PPML) technique that protects private data with sensitive information in ML, we propose a new task-specific adaptive differential privacy (DP) technique for structured data. The main idea of the proposed DP method is to adaptively calibrate the amount and distribution of random noise applied to each attribute according to the feature importance for the specific tasks of ML models and different types of data. From experimental results under various datasets, tasks of ML models, different DP mechanisms, and so on, we evaluate the effectiveness of the proposed task-specific adaptive DP method. Thus, we show that the proposed task-specific adaptive DP technique satisfies the model-agnostic property to be applied to a wide range of ML tasks and various types of data while resolving the privacy–utility trade-off problem.

show abstract

“…To generate high utility data whose distribution is similar to the original data distribution, these DP methods guarantee DP with large values of

. As described in [ 7 ], large values of

cause to expose sensitive information under privacy attacks targeting data. On the other hand, while generating data with high privacy guarantees, these DP methods decrease the utility of data.…”

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Task-Specific Adaptive Differential Privacy Method for Structured Data

Utaliyeva

Shin

Choi

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Nevertheless, we also specifically examine the impact of updating the NER model with different phrases (variants of the surrounding sentence) containing the new named entity in Section 4.3. Note that unknown surrounding sentence variant of the attack makes it more of an attribute inference attack [63,66] rather than a membership inference attack.…”

Section: Threat Modelmentioning

confidence: 99%

“…We also demonstrate that it is possible to infer words close to the secret words, which helps an attacker find the target password even when it is not in her initial dictionary of possible passwords. This is known as attribute inference in the literature [63,66]. Next we demonstrate our attack on a practical use-case of document redaction, whereby the NER model is updated on a publicly available medical dataset, which we assume to be a 1 See Section 3 for the precise definition of the input and output.…”

Section: Introductionmentioning

confidence: 99%

Unintended Memorization and Timing Attacks in Named Entity Recognition Models

Ali

Zhao

Asghar

et al. 2023

PoPETs

View full text Add to dashboard Cite

Named entity recognition models (NER), are widely used for identifying named entities (e.g., individuals, locations, and other information) in text documents. Machine learning based NER models are increasingly being applied in privacy-sensitive applications that need automatic and scalable identification of sensitive information to redact text for data sharing. In this paper, we study the setting when NER models are available as a black-box service for identifying sensitive information in user documents and show that these models are vulnerable to membership inference on their training datasets. With updated pre-trained NER models from spaCy, we demonstrate two distinct membership attacks on these models. Our first attack capitalizes on unintended memorization in the NER's underlying neural network, a phenomenon NNs are known to be vulnerable to. Our second attack leverages a timing side-channel to target NER models that maintain vocabularies constructed from the training data. We show that different functional paths of words within the training dataset in contrast to words not previously seen have measurable differences in execution time. Revealing membership status of training samples has clear privacy implications. For example, in text redaction, sensitive words or phrases to be found and removed, are at risk of being detected in the training dataset. Our experimental evaluation includes the redaction of both password and health data, presenting both security risks and a privacy/regulatory issues. This is exacerbated by results that indicate memorization after only a single phrase. We achieved a 70% AUC in our first attack on a text redaction use-case. We also show overwhelming success in the second timing attack with an 99.23% AUC. Finally we discuss potential mitigation approaches to realize the safe use of NER models in light of the presented privacy and security implications of membership inference attacks.

show abstract

“…MIAs for ML: An MIA was introduced in the ML setting by Shokri et al [63] (their attack is called NN-based attack) and its formal definition was given by Yeom et al [83]. The relationship between MIA and other notions such as DP [15,16] and attribute inference attacks [19,20] were shown by Yeom et al [83] and Zhao et al [86].…”

Section: Related Workmentioning

confidence: 99%

Knowledge Cross-Distillation for Membership Privacy

Chourasia¹,

Enkhtaivan²,

Ito³

et al. 2021

Preprint

View full text Add to dashboard Cite

A membership inference attack (MIA) poses privacy risks on the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The stateof-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data to protect but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medical and financial, the availability of public data is not obvious. Moreover, a trivial method to generate the public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs using knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable with those of DMP for the benchmark tabular datasets used in MIA researches, Purchase100 and Texas100, and our defense has much better privacy-utility trade-off than those of the existing defenses without using public data for image dataset CIFAR10.

show abstract

On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models

Cited by 17 publications

References 15 publications

Task-Specific Adaptive Differential Privacy Method for Structured Data

Task-Specific Adaptive Differential Privacy Method for Structured Data

Unintended Memorization and Timing Attacks in Named Entity Recognition Models

Knowledge Cross-Distillation for Membership Privacy

Contact Info

Product

Resources

About