GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation

Abstract

Graph Neural Networks (GNNs) are powerful models designed for graph data that learn node representation by recursively aggregating information from each node's local neighborhood. However, despite their state-of-the-art performance in predictive graph-based applications, recent studies have shown that GNNs can raise significant privacy concerns when graph data contain sensitive information. As a result, in this paper, we study the problem of learning GNNs with Differential Privacy (DP). We propose GAP, a novel differentially private GNN that safeguards the privacy of nodes and edges using aggregation perturbation, i.e., adding calibrated stochastic noise to the output of the GNN's aggregation function, which statistically obfuscates the presence of a single edge (edge-level privacy) or a single node and all its adjacent edges (node-level privacy). To circumvent the accumulation of privacy cost at every forward pass of the model, we tailor the GNN architecture to the specifics of private learning. In particular, we first precompute private aggregations by recursively applying neighborhood aggregation and perturbing the output of each aggregation step. Then, we privately train a deep neural network on the resulting perturbed aggregations for any node-wise classification task. A major advantage of GAP over previous approaches is that we guarantee edgelevel and node-level DP not only for training, but also at inference time with no additional costs beyond the training's privacy budget. We theoretically analyze the formal privacy guarantees of GAP using Rényi DP. Empirical experiments conducted over three real-world graph datasets demonstrate that GAP achieves a favorable privacy-accuracy trade-off and significantly outperforms existing approaches.

Certified Graph Unlearning

“…Remarkably, we can still obtain competitive performance with SGC Retraining when we require to be as small as 1. In contrast, one needs at least ≥ 5 to unlearn even one node or edge by leveraging state-of-the-art DP-GNNs [23,20] for reasonable performance, albeit our tested datasets are different. This shows the benefit of our certified graph unlearning method as opposed to both retraining from scratch and DP-GNNs.…”

“…Machine unlearning can therefore be viewed as a means to trade-off between performance and computational cost, with complete retraining and DP on two different ends of the spectrum [2]. Several recent works proposed DP-GNNs [20,21,22,23] -however, even for unlearning one single node or edge, these methods require a high "privacy budget" to learn with sufficient accuracy.…”

Privacy and Transparency in Graph Machine Learning: A Unified Perspective

“…Compared to general deep learning models, GraphML is more vulnerable to privacy risks as they incorporate not only the node features/labels but also the graph structure [20]. Privacy preserving techniques for graph models are mainly based on differential privacy [20,29,30] and adversarial training frameworks [11,17,16].…”

“…Moreover, due to the correlated nature of the graph data, privacy-preserving mechanisms on graph models need to focus on several aspects such as node privacy, edge privacy, and attribute privacy [30]. This leads to more complex privacy-preserving mechanisms, which results in a further loss of transparency.…”

“…Consequently, an increasing number of works are focussing on explaining [37,7,18,35] the decisions of black box GraphML models in a post-hoc manner, designing interpretability by design models [40,26,43] as well as privacy preserving techniques for real world deployments of graph models [21,29,30].…”