Node-Level Differentially Private Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) are a popular technique for modelling graphstructured data that compute node-level representations via aggregation of information from the local neighborhood of each node. However, this aggregation implies increased risk of revealing sensitive information, as a node can participate in the inference for multiple nodes. This implies that standard privacy preserving machine learning techniques, such as differentially private stochastic gradient descent (DP-SGD) -which are designed for situations where each data point participates in the inference for one point only -either do not apply, or lead to inaccurate solutions. In this work, we formally define the problem of learning 1-layer GNNs with node-level privacy, and provide an algorithmic solution with a strong differential privacy guarantee. Even though each node can be involved in the inference for multiple nodes, by employing a careful sensitivity analysis and a non-trivial extension of the privacy-by-amplification technique, our method is able to provide accurate solutions with solid privacy parameters. Empirical evaluation on standard benchmarks demonstrates that our method is indeed able to learn accurate privacy preserving GNNs, while still outperforming standard non-private methods that completely ignore graph information.

Certified Graph Unlearning

“…Remarkably, we can still obtain competitive performance with SGC Retraining when we require to be as small as 1. In contrast, one needs at least ≥ 5 to unlearn even one node or edge by leveraging state-of-the-art DP-GNNs [23,20] for reasonable performance, albeit our tested datasets are different. This shows the benefit of our certified graph unlearning method as opposed to both retraining from scratch and DP-GNNs.…”

“…Machine unlearning can therefore be viewed as a means to trade-off between performance and computational cost, with complete retraining and DP on two different ends of the spectrum [2]. Several recent works proposed DP-GNNs [20,21,22,23] -however, even for unlearning one single node or edge, these methods require a high "privacy budget" to learn with sufficient accuracy.…”

Locally Private Graph Neural Networks

“…We use GAP's official implementation on GitHub 1 and follow the same experimental setup as reported in the original paper. We do not include other available differentially private GNN approaches as they either: (i) are outperformed by GAP (e.g., [8,44]) or (ii) have different problem settings (e.g., [34,38]) that make them not directly comparable to our method.…”

“…Nevertheless, their approach relies on public graph data and may not be applicable in all situations. Daigavane et al [8] extend the standard DP-SGD algorithm and privacy amplification by subsampling to bounded-degree graph data to achieve node-level DP, but their method fails to provide inference privacy. Finally, Sajadmanesh et al [39] propose GAP, a private GNN learning framework that provides both edge-level and node-level privacy guarantees using the aggregation perturbation approach.…”

“…To address the privacy concerns associated with GNNs, researchers have recently studied differential privacy (DP), a well-established mathematical framework that provides strong privacy guarantees, usually by adding random noise to the data [9,10]. However, applying DP to GNNs is very challenging due to the complex structural connectivity of graphs, rendering traditional private learning methods, such as differentially private stochastic gradient descent (DP-SGD) [1], infeasible [3,8,39]. Recently, the aggregation perturbation (AP) approach [39] has emerged as a state-of-the-art technique for ensuring DP in GNNs.…”