GIF: A General Graph Unlearning Strategy via Influence Function

Abstract

With the greater emphasis on privacy and security in our society, the problem of graph unlearning -revoking the influence of specific data on the trained GNN model, is drawing increasing attention. However, ranging from machine unlearning to recently emerged graph unlearning methods, existing efforts either resort to retraining paradigm, or perform approximate erasure that fails to consider the inter-dependency between connected neighbors or imposes constraints on GNN structure, therefore hard to achieve satisfying performance-complexity trade-offs.In this work, we explore the influence function tailored for graph unlearning, so as to improve the unlearning efficacy and efficiency for graph unlearning. We first present a unified problem formulation of diverse graph unlearning tasks w.r.t. node, edge, and feature. Then, we recognize the crux to the inability of traditional influence function for graph unlearning, and devise Graph Influence Function (GIF), a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a 𝜖-mass perturbation in deleted data. The idea is to supplement the objective of the traditional influence function with an additional loss term of the influenced neighbors due to the structural dependency. Further deductions on the closed-form solution of parameter changes provide a better understanding of the unlearning mechanism. We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify the superiority of GIF for diverse graph unlearning tasks in terms of unlearning efficacy, model utility, and unlearning efficiency. Our implementations are available at https://github.com/wujcan/GIF-torch/.

GraphGuard: Detecting and Counteracting Training Data Misuse in Graph Neural Networks

“…This presents significant challenges compared to previous unlearning studies. In particular, most existing processes [11,13,14,64] require that the entities performing the unlearning can access the unlearning samples so that these samples guide which information to forget. In the absence of these samples, the server remains clueless about which specific knowledge should be expunged from the GNN model undergoing unlearning, and making ordinary unlearning methods inapplicable.…”

“…Despite its ingenuity, the design required an additional block where the unlearning data was stored, reducing its practicality in the MLaaS setting. Wu et al [64] devised an unlearning strategy, defining an influence function known as the Graph Influence Function (GIF), which is considered an additional loss term for influenced neighbors, facilitating the unlearning of the graph structure. Moreover, Pan et al [48] suggested an unlearning technique for GNNs that employed the Graph Scattering Transform, providing provable performance guarantees.…”

“…[6] [11] Unlearning-(Others) [8], [60], [16], [31], [82] [14], [13], [64], [48] GraphGuard (Our Design) data owners have to transfer their exact training samples to the cloud for membership inference, where they may be hesitant to do so due to the sensitivity of their graph data, intellectual property considerations, and data usage agreements. 2) Mitigating data misuse.…”

“…2) Mitigating data misuse. Most existing studies concentrate on mitigating data misuse through unlearning [6,11,14,64], which involves removing the influence of certain training samples from the trained model. However, they call for particular function blocks in their GNN architecture or GNN training process to ensure unlearning, which is incompatible with general GNN models deployed on MLaaS platforms.…”

On the Effectiveness of Unlearning in Session-Based Recommendation

“…[11] modified the SISA algorithm to work for sequences of deletion requests. Another kind of method for exact unlearning involves selective influence estimators [35], which calculate the influence of the unlearning samples on the model parameters. Although such influence-based methods are effective in terms of privacy preservation, the high computational cost limits their application for real-world scenarios [39].…”