On the Privacy Risks of Model Explanations

Shokri, Reza; Strobel, Martin; Zick, Yair

doi:10.1145/3461702.3462533

Cited by 67 publications

(99 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Providing adequate privacy preservation for EI applications is an area with open research challenges. To preserve the privacy of client's data, different enabling technology are utilised with or without cryptographic techniques, perturbation techniques, and anonymisation techniques [227,255]. On the one hand, these technologies and techniques provide a means to safeguard the client's data better but simultaneously struggles to maintain the effectiveness (accuracy in case of classification problem) of the AI model [7,65,105].…”

Section: Privacy-preservation In Edge-aimentioning

confidence: 99%

Enabling Deep Learning for All-in EDGE paradigm

Joshi¹,

Hasanuzzaman²,

Thapa³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning-based models have been widely investigated, and they have demonstrated significant performance on non-trivial tasks such as speech recognition, image processing, and natural language understanding. However, this is at the cost of substantial data requirements. Considering the widespread proliferation of edge devices (e.g., Internet of Things devices) over the last decade, Deep Learning in the edge paradigm, such as device-cloud integrated platforms, is required to leverage its superior performance. Moreover, it is suitable from the data requirements perspective in the edge paradigm because the proliferation of edge devices has resulted in an explosion in the volume of generated and collected data. However, there are difficulties due to other requirements such as high computation, high latency, and high bandwidth caused by Deep Learning applications in real-world scenarios. In this regard, this survey paper investigates Deep Learning at the edge, its architecture, enabling technologies, and model adaption techniques, where edge servers and edge devices participate in deep learning training and inference. For simplicity, we call this paradigm the All-in EDGE paradigm. Besides, this paper presents the key performance metrics for Deep Learning at the All-in EDGE paradigm to evaluate various deep learning techniques and choose a suitable design. Moreover, various open challenges arising from the deployment of Deep Learning at the All-in EDGE paradigm are identified and discussed.

show abstract

Section: Privacy-preservation In Edge-aimentioning

confidence: 99%

Enabling Deep Learning for All-in EDGE paradigm

Joshi¹,

Hasanuzzaman²,

Thapa³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The goal of building explainability is to provide a global understanding of how the FL server makes decisions and how they impact each client's interest in order to build trust between the two parties. However, research in explainability in FL must be framed within the context of privacy preservation so as not to conflict with the primary goal of FL [48].…”

Section: Trust Building Through Explainabilitymentioning

confidence: 99%

Towards Fairness-Aware Federated Learning

Shi¹,

Han²,

Leung³

2021

Preprint

View full text Add to dashboard Cite

Recent advances in Federated Learning (FL) have brought large-scale machine learning opportunities for massive distributed clients with performance and data privacy guarantees. However, most current works only focus on the interest of the central controller in FL, and ignore the interests of clients. This may result in unfairness which discourages clients from actively participating in the learning process and damages the sustainability of the whole FL system. Therefore, the topic of ensuring fairness in an FL is attracting a great deal of research interest. In recent years, diverse Fairness-Aware FL (FAFL) approaches have been proposed in an effort to achieve fairness in FL from different viewpoints. However, there is no comprehensive survey which helps readers gain insight into this interdisciplinary field. This paper aims to provide such a survey. By examining the fundamental and simplifying assumptions, as well as the notions of fairness adopted by existing literature in this field, we propose a taxonomy of FAFL approaches covering major steps in FL, including client selection, optimization, contribution evaluation and incentive distribution. In addition, we discuss the main metrics for experimentally evaluating the performance of FAFL approaches, and suggest some promising future research directions.

show abstract

“…In addition to classification and generative models, MIA have been investigated on various domains, including embedding models [91], regression models [28], sequence-to-sequence models [34,93], image segmentation [31,86], transfer learning models [11,58,125], algorithmic fairness [10], model explanations [88,89], adversarial machine learning [95,96], and graph neural network [30,74]. Song and Raghunathan [91] investigate the membership risks on embedding models and demonstrate a simple threshold attack can achieve a 30% improvement in attack accuracy over random guessing on Word2Vec [67], Fast-Text [7], Glove [76], LSTM [35], and Transformer [106] models.…”

Section: Membership Inference Attacks On Different Domainsmentioning

confidence: 99%

“…Liew and Takahashi [58] present that in the application of transfer learning on face recognition, an adversary can successfully implement MIA on the teacher model even when it only accesses the student models. Shokri et al [88,89] show that releasing transparency reports of ML models can result in membership privacy risks. Song et al [95,96] find that the adversarially trained model is more vulnerable to MIA.…”

Section: Membership Inference Attacks On Different Domainsmentioning

confidence: 99%

Membership Inference Attacks on Machine Learning: A Survey

Hu¹,

Salčić²,

Sun³

et al. 2021

Preprint

View full text Add to dashboard Cite

Membership inference attack aims to identify whether a data sample was used to train a machine learning model or not. It can raise severe privacy risks as the membership can reveal an individual's sensitive information. For example, identifying an individual's participation in a hospital's health analytics training set reveals that this individual was once a patient in that hospital. Membership inference attacks have been shown to be effective on various machine learning models, such as classification models, generative models, and sequenceto-sequence models. Meanwhile, many methods are proposed to defend such a privacy attack. Although membership inference attack is an emerging and rapidly growing research area, there is no comprehensive survey on this topic yet. In this paper, we bridge this important gap in membership inference attack literature. We present the first comprehensive survey of membership inference attacks. We summarize and categorize existing membership inference attacks and defenses and explicitly present how to implement attacks in various settings. Besides, we discuss why membership inference attacks work and summarize the benchmark datasets to facilitate comparison and ensure fairness of future work. Finally, we propose several possible directions for future research and possible applications relying on reviewed works. CCS Concepts: • Security and privacy → Privacy protections.

show abstract

On the Privacy Risks of Model Explanations

Cited by 67 publications

References 10 publications

Enabling Deep Learning for All-in EDGE paradigm

Enabling Deep Learning for All-in EDGE paradigm

Towards Fairness-Aware Federated Learning

Membership Inference Attacks on Machine Learning: A Survey

Contact Info

Product

Resources

About