Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling

Meng, Chuizheng; Rambhatla, Sirisha; Liu, Yan

doi:10.1145/3447548.3467371

Cited by 79 publications

(29 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hsieh et al [26] also propose an adversarial defense mechanism against attribute inference attacks on GNNs by maintaining the accuracy of target label classification and reducing the accuracy of private label classification. There is also increasing trend in using federated and split learning to address the privacy issues of GNNs in distributed learning settings [5,6,23,34,35,39,44,53,56,60,66,67]. However, none of the aforementioned works employ the notion of DP, and thus they do not provide provable privacy guarantees.…”

Section: Privacy-preserving Gnnsmentioning

confidence: 99%

GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation

Sajadmanesh¹,

Shamsabadi²,

Bellet³

et al. 2022

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) are powerful models designed for graph data that learn node representation by recursively aggregating information from each node's local neighborhood. However, despite their state-of-the-art performance in predictive graph-based applications, recent studies have shown that GNNs can raise significant privacy concerns when graph data contain sensitive information. As a result, in this paper, we study the problem of learning GNNs with Differential Privacy (DP). We propose GAP, a novel differentially private GNN that safeguards the privacy of nodes and edges using aggregation perturbation, i.e., adding calibrated stochastic noise to the output of the GNN's aggregation function, which statistically obfuscates the presence of a single edge (edge-level privacy) or a single node and all its adjacent edges (node-level privacy). To circumvent the accumulation of privacy cost at every forward pass of the model, we tailor the GNN architecture to the specifics of private learning. In particular, we first precompute private aggregations by recursively applying neighborhood aggregation and perturbing the output of each aggregation step. Then, we privately train a deep neural network on the resulting perturbed aggregations for any node-wise classification task. A major advantage of GAP over previous approaches is that we guarantee edgelevel and node-level DP not only for training, but also at inference time with no additional costs beyond the training's privacy budget. We theoretically analyze the formal privacy guarantees of GAP using Rényi DP. Empirical experiments conducted over three real-world graph datasets demonstrate that GAP achieves a favorable privacy-accuracy trade-off and significantly outperforms existing approaches.

show abstract

Section: Privacy-preserving Gnnsmentioning

confidence: 99%

GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation

Sajadmanesh¹,

Shamsabadi²,

Bellet³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Temporal modeling in federated learning and, more specifically, in FSSL is still relatively unexplored. Only a few applications exist such as traffic flow forecasting [42], human activity recognition [43], [44], audio recognition [45], and machine fault diagnosis [46].…”

Section: B Federated Learning With Unlabeled Datamentioning

confidence: 99%

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

Kassem¹,

Alapatt²,

Mascagni³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advancements in deep learning methods bring computer-assistance a step closer to fulfilling promises of safer surgical procedures. However, the generalizability of such methods is often dependent on training on diverse datasets from multiple medical institutions, which is a restrictive requirement considering the sensitive nature of medical data. Recently proposed collaborative learning methods such as Federated Learning (FL) allow for training on remote datasets without the need to explicitly share data. Even so, data annotation still represents a bottleneck, particularly in medicine and surgery where clinical expertise is often required. With these constraints in mind, we propose FedCy, a federated semi-supervised learning (FSSL) method that combines FL and self-supervised learning to exploit a decentralized dataset of both labeled and unlabeled videos, thereby improving performance on the task of surgical phase recognition. By leveraging temporal patterns in the labeled data, FedCy helps guide unsupervised training on unlabeled data towards learning taskspecific features for phase recognition. We demonstrate significant performance gains over state-of-the-art FSSL methods on the task of automatic recognition of surgical phases using a newly collected multi-institutional dataset of laparoscopic cholecystectomy videos. Furthermore, we demonstrate that our approach also learns more generalizable features when tested on data from an unseen domain.

show abstract

“…[66] propose a hybrid of federated and meta learning to solve the semi-supervised graph node classification problem in decentralized social network datasets. [46] uses an edge-cloud partitioned GNN model for spatio-temporal traffic forecasting tasks over sensor networks. The previous works do not consider graph learning in a decentralized setting.…”

Section: Flmentioning

confidence: 99%

SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks

He,

Ceyani,

Balasubramanian

et al. 2021

Preprint

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) are the first choice methods for graph machine learning problems thanks to their ability to learn state-of-the-art level representations from graph-structured data. However, centralizing a massive amount of real-world graph data for GNN training is prohibitive due to user-side privacy concerns, regulation restrictions, and commercial competition. Federated Learning is the de-facto standard for collaborative training of machine learning models over many distributed edge devices without the need for centralization. Nevertheless, training graph neural networks in a federated setting is vaguely defined and brings statistical and systems challenges. This work proposes SpreadGNN, a novel multi-task federated training framework capable of operating in the presence of partial labels and absence of a central server for the first time in the literature. SpreadGNN extends federated multi-task learning to realistic serverless settings for GNNs, and utilizes a novel optimization algorithm with a convergence guarantee, Decentralized Periodic Averaging SGD (DPA-SGD), to solve decentralized multi-task learning problems. We empirically demonstrate the efficacy of our framework on a variety of non-I.I.D. distributed graph-level molecular property prediction datasets with partial labels. Our results show that SpreadGNN outperforms GNN models trained over a central server-dependent federated learning system, even in constrained topologies. The source code is publicly available at https://github.com/FedML-AI/SpreadGNN.

show abstract

Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling

Cited by 79 publications

References 15 publications

GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation

GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation

Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phases

SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks

Contact Info

Product

Resources

About