Federated Learning on Non-IID Graphs via Structural Knowledge Sharing

Abstract

Graph neural networks (GNNs) have shown their superiority in modeling graph data. Owing to the advantages of federated learning, federated graph learning (FGL) enables clients to train strong GNN models in a distributed manner without sharing their private data. A core challenge in federated systems is the non-IID problem, which also widely exists in real-world graph data. For example, local data of clients may come from diverse datasets or even domains, e.g., social networks and molecules, increasing the difficulty for FGL methods to capture commonly shared knowledge and learn a generalized encoder. From real-world graph datasets, we observe that some structural properties are shared by various domains, presenting great potential for sharing structural knowledge in FGL. Inspired by this, we propose FedStar, an FGL framework that extracts and shares the common underlying structure information for inter-graph federated learning tasks. To explicitly extract the structure information rather than encoding them along with the node features, we define structure embeddings and encode them with an independent structure encoder. Then, the structure encoder is shared across clients while the feature-based knowledge is learned in a personalized way, making FedStar capable of capturing more structure-based domain-invariant information and avoiding feature misalignment issues. We perform extensive experiments over both cross-dataset and cross-domain non-IID FGL settings, demonstrating the superiority of FedStar.

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning

“…Especially aiming to address the communication challenge in minimax federated framework (e.g., GAN), FedGDA-GT [169] combines gradient tacking with federated gradient descent ascent framework, showcasing linear convergence with constant stepsizes to a globalapproximation solution. For federated node embedding problems in graph machine learning [170], Pan and Zhu [171] proposed a random-walk-based algorithm featuring a sequence encoder for privacy preservation and a two-hop neighbor predictor, effectively reducing communication costs.…”

A Social Media Knowledge Retrieval Method Based on Knowledge Demands and Knowledge Supplies

“…Manias et al (2023) [18] improved sentiment knowledge acquisition by introducing multilingual BERT-based classifiers, which enhances the accuracy of sentiment analysis. Tan et al (2023) [19] introduced an FGL framework to strong GNNs that extracts the common underlying structure knowledge, demonstrating its superiority over existing methods in cross-dataset and cross-domain non-IID settings. However, the aforementioned classical methods do not consider the creation of a hierarchical and systematic knowledge organization, which can enhance the efficiency of knowledge retrieval and improve the user's search experience.…”