Machine Unlearning

Bourtoule, Lucas; Chandrasekaran, Varun; Choquette-Choo, Christopher A.; Jia, Hengrui; Travers, Adelin; Zhang, Baiwu; Lie, David; Papernot, Nicolas

doi:10.1109/sp40001.2021.00019

Cited by 178 publications

(125 citation statements)

References 33 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…One solution has been to design new models or adapt existing ones with unlearning efficiency in mind. Examples of such models include ensembles of randomized decision trees (Schelter, Grafberger, and Dunning, 2021), variants of k-means clustering (Ginart et al, 2019), and a modeling framework called SISA that takes advantage of data sharding and caching (Bourtoule et al, 2021).…”

Section: Resultsmentioning

confidence: 99%

“…Gupta et al (2021) considered an adaptive unlearning setting, where erasure requests may depend on previously published models. They also demonstrated an attack on the SISA algorithm (Bourtoule et al, 2021). Sommer et al (2020) considered verification of unlearning-an important privacy feature for users who cannot trust organizations to erase their data.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Hard to Forget: Poisoning Attacks on Certified Machine Unlearning

Marchant¹,

Rubinstein²,

Alfeld³

2021

Preprint

View full text Add to dashboard Cite

The right to erasure requires removal of a user's information from data held by organizations, with rigorous interpretations extending to downstream products such as learned models. Retraining from scratch with the particular user's data omitted fully removes its influence on the resulting model, but comes with a high computational cost. Machine "unlearning" mitigates the cost incurred by full retraining: instead, models are updated incrementally, possibly only requiring retraining when approximation errors accumulate. Rapid progress has been made towards privacy guarantees on the indistinguishability of unlearned and retrained models, but current formalisms do not place practical bounds on computation. In this paper we demonstrate how an attacker can exploit this oversight, highlighting a novel attack surface introduced by machine unlearning. We consider an attacker aiming to increase the computational cost of data removal. We derive and empirically investigate a poisoning attack on certified machine unlearning where strategically designed training data triggers complete retraining when removed.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Hard to Forget: Poisoning Attacks on Certified Machine Unlearning

Marchant¹,

Rubinstein²,

Alfeld³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, Purchase dataset (with 197, 324 records) does not contain any class labels. Following existing works [21], [31], [32], we adopt an unsupervised clustering algorithm to assign each data record with a class label. In this paper, we cluster the records in Purchase dataset into 2 classes.…”

Section: Performance Evaluationmentioning

confidence: 99%

“…Recently, Bourtoule et al [21] introduce a more general algorithm named SISA, which takes advantage of sharding and slicing during the training. Specifically, to remove a specific sample, retraining is only performed in the shard that contains this sample.…”

Section: Related Work a Machine Unlearningmentioning

confidence: 99%

“…Existing studies on unlearning in the context of ML (a.k.a. machine unlearning) [19], [20], [21], [22] cannot eliminate the influence of the unlearned client's data on the global FL model either, mainly due to the inherent distinction in the way how FL and ML learn from data. In particular, FL employs the iterative training involving multiple rounds of training, where the clients' initial model for each round of training is obtained from the previous round and contains the information of all clients including the one that requires to be unlearned.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Federated Unlearning

Liu,

Ma,

Yang

et al. 2020

Preprint

View full text Add to dashboard Cite

Data removal from machine learning models has been paid more attentions due to the demands of the "right to be forgotten" and countering data poisoning attacks. In this paper, we frame the problem of federated unlearning, a postprocess operation of the federated learning models to remove the influence of the specified training sample(s). We present FedEraser, the first federated unlearning methodology that can eliminate the influences of a federated client's data on the global model while significantly reducing the time consumption used for constructing the unlearned model. The core idea of FedEraser is to trade the central server's storage for unlearned model's construction time. In particular, FedEraser reconstructs the unlearned model by leveraging the historical parameter updates of federated clients that have been retained at the central server during the training process of FL. A novel calibration method is further developed to calibrate the retained client updates, which can provide a significant speed-up to the reconstruction of the unlearned model. Experiments on four realistic datasets demonstrate the effectiveness of FedEraser, with an expected speed-up of 4× compared with retraining from the scratch.

show abstract

Contesting efficacy: Tensions between risk and inclusion in computer vision technology

Scheuerman

2024

Future Humanities

View full text Add to dashboard Cite

| INTRODUCTION AND BACKGROUNDMachine learning (ML) methods are now commonly used to make automated predictions about human beings-their lives and their characteristics. Vast amounts of individual data are aggregated to make predictions about people's shopping preferences, health status, or likelihood to recommit a crime. Computer vision, an ML task for training a computer to metaphorically 'see' specific objects, is a pertinent domain for examining the interaction between ML and human identity. Facial analysis (FA), a subset of computer vision trained to complete tasks like facial classification and facial recognition, is trained to read visual data to make classifications about innate human identities. Identities like age (Lin et al., 2006), gender (Khan et al., 2013), ethnicity (Lu & Jain, 2004) and even sexual orientation (Wang & Kosinski, 2017). Often, decisions about identity characteristics are made without explicit user input-or even user knowledge. Users, effectively, become 'targets' of the system, having no ability to contest these classifications. Surrounding these identity classifications are concerns about bias (e.g., Buolamwini & Gebru, 2018), representation (e.g., Hamidi et al., 2018;Keyes, 2018) and the embracing of pseudoscientific practices like physiognomy (e.g., Agüera y Arcas et al., 2017).In this short paper, I present several considerations for contestability for computer vision. By contestability, I refer to the agency that an individual has to contest the inputs and outputs of a computer vision system-including how one's data is collected, defined and used. I specifically focus on one identity trait for which to ground consideration: gender. Gender is a salient characteristic to consider given that criticisms of computer vision have stemmed from concerns of both sexism and cissexism, discrimination against transgender and nonbinary communities (Hibbs, 2014). Gender in computer vision has largely been presented as binary (i.e., male vs. female) and has been exclusive of genders beyond the cisgender norm (e.g., in automatic gender recognition (AGR) systems that classify gender explicitly [Hamidi et al., 2018;

show abstract

Machine Unlearning

Cited by 178 publications

References 33 publications

Hard to Forget: Poisoning Attacks on Certified Machine Unlearning

Hard to Forget: Poisoning Attacks on Certified Machine Unlearning

Federated Unlearning

Contesting efficacy: Tensions between risk and inclusion in computer vision technology

Contact Info

Product

Resources

About