Graph Unlearning

Abstract

Machine unlearning is a process of removing the impact of some training data from the machine learning (ML) models upon receiving removal requests. While straightforward and legitimate, retraining the ML model from scratch incurs a high computational overhead. To address this issue, a number of approximate algorithms have been proposed in the domain of image and text data, among which SISA is the state-of-the-art solution. It randomly partitions the training set into multiple shards and trains a constituent model for each shard. However, directly applying SISA to the graph data can severely damage the graph structural information, and thereby the resulting ML model utility. In this paper, we propose GraphEraser, a novel machine unlearning framework tailored to graph data. Its contributions include two novel graph partition algorithms and a learning-based aggregation method. We conduct extensive experiments on five real-world graph datasets to illustrate the unlearning efficiency and model utility of GraphEraser. It achieves 2.06× (small dataset) to 35.94× (large dataset) unlearning time improvement. On the other hand, GraphEraser achieves up to 62.5% higher F1 score and our proposed learning-based aggregation method achieves up to 112% higher F1 score. 1 CCS CONCEPTS• Security and privacy; • Computing methodologies → Machine learning;

GIF: A General Graph Unlearning Strategy via Influence Function

“…• Unlearning Efficiency: We record the running time to reflect the unlearning efficiency across different unlearning algorithms. • Model Utility: As in GraphEraser [6], we use F1 score -the harmonic average of precision and recall -to measure the utility. • Unlearning Efficacy: Due to the nonconvex nature of graph unlearning, it is intractable to measure the unlearning efficacy through the lens of model parameters.…”

“…It can achieve good model utility but falls short in unlearning efficiency. • GraphEraser [6]. This is an efficient retraining method.…”

“…All the GNN models are implemented with the PyTorch Geometric library. Following the settings of GraphEraser [6], for each GNN model, we fix the number of layers to 2, and train each model for 100 epochs. For GraphEraser, as suggested by the paper [6], we partition the three datasets, Cora, Citeseer, and Physics, with two proposed methods, BLPA and BKEM, into 20, 20, and 50 shards respectively.…”

“…GraphEditor [9] is a very recent work that supports exact graph unlearning free from shard model retraining, but is restricted to linear GNN structure under Ridge regression formulation. • Another line [8,13,14,23,24,30] resorts to gradient analysis techniques instead to approximate the unlearning process, so as to avoid retraining sub-models from scratch. Among them, influence function [20] is a promising proxy to estimate the parameter changes caused by a sample removal, which is on up-weighting the individual loss w.r.t.…”

“…• Considering the unlearning efficiency, some follow-on studies [1,3,5,6,9] first divide the whole training data into multiple disjoint shards, and then train one sub-model for each shard.…”