FedKC: Federated Knowledge Composition for Multilingual Natural Language Understanding

Wang, Haoyu; Zhao, Handong; Wang, Yaqing; Yu, Tong; Gu, Jiuxiang; Gao, Jing

doi:10.1145/3485447.3511988

Cited by 7 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the principle of knowledge distillation, using the prediction output of the model can still transfer the knowledge learned by the model through the training data, so we can quote the idea of knowledge distillation and use the prediction output of the client model to replace the model parameters for knowledge transfer. Some studies [15][16][17] have introduced the training method of knowledge distillation to improve the original federated learning, replacing the model parameters as the interactive information between the client and the server with the client prediction output, and showed that the federated learning using this method can also obtain acceptable performance. However, in these works [15][16][17], the aggregation method of Federated learning has not been changed.…”

Section: Knowledge Distillationmentioning

confidence: 99%

“…Some studies [15][16][17] have introduced the training method of knowledge distillation to improve the original federated learning, replacing the model parameters as the interactive information between the client and the server with the client prediction output, and showed that the federated learning using this method can also obtain acceptable performance. However, in these works [15][16][17], the aggregation method of Federated learning has not been changed. It is still the weighted average of the output results uploaded by each client, and whether the effect of the aggregation method is really the most effective has not been explained too much.…”

Section: Knowledge Distillationmentioning

confidence: 99%

See 1 more Smart Citation

Federated learning for feature-fusion based requirement classification

Wang,

Liu,

Zhang

et al. 2023

Cluster Comput

View full text Add to dashboard Cite

With the increasing complexity of products, the utilization of automated requirements classification in the process of requirement engineering is of positive significance to improve the efficiency of product development. There have been many experiments on the requirements classification based on artificial intelligence. However, many studies only use a single feature in semantic features or statistical features to classify requirements and put forward the problem of small-scale requirement datasets caused by the confidentiality of requirement data, which limits the application research of requirement classification in a large number of fields.Therefore, we propose a requirement classification model that integrates semantic and statistical features, and design a federated learning framework based on knowledge distillation without centralized servers in a ring architecture. In this paper, we extract a large amount of requirement data from a large number of requirement documents, and conduct experiments in the centralized learning environment and the federated learning environment respectively.The final experimental results show that our method can maintain the confidentiality of the requirement data and improve the communication efficiency in the process of model training with a slight decrease in the effect of requirement classification compared with centralized learning.

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%

Section: Knowledge Distillationmentioning

confidence: 99%

Federated learning for feature-fusion based requirement classification

Wang,

Liu,

Zhang

et al. 2023

Cluster Comput

View full text Add to dashboard Cite

show abstract

“… 15 Multilingual FL has been recently explored in different language tasks, as it provides an interesting and natural setting to examine non-IID data, of which different languages are an obvious application. 16 , 17 , 18…”

Section: Introductionmentioning

confidence: 99%

Federated learning for privacy-preserving depression detection with multilingual language models in social media posts

Khalil,

Tawfik,

Spruit

2024

Patterns

View full text Add to dashboard Cite

A comprehensive survey of federated transfer learning: challenges, methods and applications

Guo,

Zhuang,

Zhang

et al. 2024

Front. Comput. Sci.

View full text Add to dashboard Cite

Federated learning (FL) is a novel distributed machine learning paradigm that enables participants to collaboratively train a centralized model with privacy preservation by eliminating the requirement of data sharing. In practice, FL often involves multiple participants and requires the third party to aggregate global information to guide the update of the target participant. Therefore, many FL methods do not work well due to the training and test data of each participant may not be sampled from the same feature space and the same underlying distribution. Meanwhile, the differences in their local devices (system heterogeneity), the continuous influx of online data (incremental data), and labeled data scarcity may further influence the performance of these methods. To solve this problem, federated transfer learning (FTL), which integrates transfer learning (TL) into FL, has attracted the attention of numerous researchers. However, since FL enables a continuous share of knowledge among participants with each communication round while not allowing local data to be accessed by other participants, FTL faces many unique challenges that are not present in TL. In this survey, we focus on categorizing and reviewing the current progress on federated transfer learning, and outlining corresponding solutions and applications. Furthermore, the common setting of FTL scenarios, available datasets, and significant related research are summarized in this survey.

show abstract

FedKC: Federated Knowledge Composition for Multilingual Natural Language Understanding

Cited by 7 publications

References 29 publications

Federated learning for feature-fusion based requirement classification

Federated learning for feature-fusion based requirement classification

Federated learning for privacy-preserving depression detection with multilingual language models in social media posts

A comprehensive survey of federated transfer learning: challenges, methods and applications

Contact Info

Product

Resources

About