Explainable machine learning models with privacy

Abstract

The importance of explainable machine learning models is increasing because users want to understand the reasons behind decisions in data-driven models. Interpretability and explainability emerge from this need to design comprehensible systems. This paper focuses on privacy-preserving explainable machine learning. We study two data masking techniques: maximum distance to average vector (MDAV) and additive noise. The former is for achieving k-anonymity, and the second uses Laplacian noise to avoid record leakage and provide a level of differential privacy. We are interested in the process of developing data-driven models that, at the same time, make explainable decisions and are privacy-preserving. That is, we want to avoid the decision-making process leading to disclosure. To that end, we propose building models from anonymized data. More particularly, data that are k-anonymous or that have been anonymized add an appropriate level of noise to satisfy some differential privacy requirements. In this paper, we study how explainability has been affected by these data protection procedures. We use TreeSHAP as our technique for explainability. The experiments show that we can keep up to a certain degree both accuracy and explainability. So, our results show that some trade-off between privacy and explainability is possible for data protection using k-anonymity and noise addition.

The privacy-explainability trade-off: unraveling the impacts of differential privacy and federated learning on attribution methods

“…As a result, methods defending the privacy of explainable models have been proposed (Montenegro et al, 2021 ; Nguyen et al, 2023 ; Pentyala et al, 2023 ). The effect of privacy-preserving training methods on explanations is far less studied (Naidu et al, 2021 ; Patel et al, 2022 ; Bozorgpanah and Torra, 2024 ). This present study tackles the lack of work investigating the overall influence of private training on feature-based explanations in deep learning for different data modalities.…”