Do Gradient Inversion Attacks Make Federated Learning Unsafe?

Abstract

Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. This capability makes it especially interesting for healthcare applications where patient and data privacy is of utmost concern. However, recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack that works for more realistic scenarios where the clients' training involves updating the Batch Normalization (BN) statistics. Furthermore, we present new ways to measure and visualize potential data leakage in FL. Our work is a step towards establishing reproducible methods of measuring data leakage in FL and could help determine the optimal tradeoffs between privacy-preserving techniques, such as differential privacy, and model accuracy based on quantifiable metrics.

Privacy preservation for federated learning in health care

“…An overview of model inversion attack implementations and defense approaches is already described in prior work. 61 Advancements in these attacks continue to be put forward, and works that demonstrate such attacks in the setting of FL for medical models that demonstrate successful approximation of hidden batch normalization statistics, for example, 62 acknowledge the importance of understanding such threats in these settings. Membership inference: membership inference is the process by which an adversary has possession of a particular data sample and is attempting to infer whether it was included in the training set of the model.…”

“…An overview of model inversion attack implementations and defense approaches is already described in prior work. 61 Advancements in these attacks continue to be put forward, and works that demonstrate such attacks in the setting of FL for medical models that demonstrate successful approximation of hidden batch normalization statistics, for example, 62 acknowledge the importance of understanding such threats in these settings.…”

“…In particular, secure and private FL has hope to greatly benefit collaborations in domains with stringent policies around data sharing, such as is the case for health care. 14 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 Some literature exists reporting on general vulnerabilities of FL 29 , 30 and even exploring privacy and security concepts related to FL, 31 , 32 , 33 albeit without providing a focus on their implications in the health-care domain. We further note a survey of open-source frameworks enabling FL, although without adequately exploring concepts of privacy.…”

Class Impression for Data-free Incremental Learning

“…To inject class-specific impression to the synthetic images without violating privacy regulations (such as re-identification), we assume the averaged image of each class xt k (k ∈ Y t ) is available to initialize the optimization of x in Eq (1). It is worth noting that, Class Impression aims to generate images following the distributions of the past class, rather than reconstructing the training data points as model inversion attacks does [24,8,6], thus Class Impression aims to meet certain privacy requirements imposed on storing medical data.…”

Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI

“…Privacy in FL and PFL Privacy is a hot and significant topic in the age of medical big data [37]. Nevertheless, previous studies showed that FL is still vulnerable to attacks, such as data poisoning attack [38], membership inference attack [39]- [41], source inference attack (SIA) [42], attribute reconstruction attack [43], and inversion attack [44]- [47], thus compromising data privacy.…”