A unified framework for multi-modal federated learning

Xiong, Baochen; Yang, Xiaoshan; Fan, Qi; Xu, Changsheng

doi:10.1016/j.neucom.2022.01.063

Cited by 38 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…T ρ+T η T ρ+T η+F ρ+F η * 100 [10], [38], [13], [36], [58], [46], [47], [28], [77], [35], [7], [79], [64], [29], [1], [85], [9], [53], [25], [51], [66], [17], [45], [50] Sensitivity (Recall)…”

Section: Measures Equation References Accuracymentioning

confidence: 99%

See 1 more Smart Citation

Role of federated learning in healthcare systems: A survey

Rana¹,

Marwaha²

2024

MFC

View full text Add to dashboard Cite

Nowadays, machine learning affects practically every industry, but the effectiveness of these systems depends on the accessibility of training data sets. Every device now produces data, and that data can serve as the foundation for upcoming technologies. Traditional machine learning systems need centralised data for their training, but the availability of valid and good amounts of data is not always possible due to various privacy risks. But federated learning can solve this issue [78]. In a federated learning (FL) environment, a model can be trained on decentralised datasets by involving a large number of participants, such as mobile devices or entire enterprises. Researchers are using this technique in various fields and getting great responses. The importance of using federated learning in the healthcare industry is highlighted in this paper since there is a wealth of data available in hospitals or electronic health records that may be used to train medical systems but cannot be shared due to privacy issues. The main contribution of this paper is to highlight the role of federated learning in the medical field. It also presents a list of frameworks available to implement federated learning models. The paper also listed the evaluation metrics used to check the efficiency of a federated learning model. Broadly used evaluation metrics are accuracy, precision, recall, and F1-score. Open issues for research in this area are also discussed at the end of this paper.

show abstract

Section: Measures Equation References Accuracymentioning

confidence: 99%

“…A "Multi-Model Federated Learning Framework (MMFed)" has been proposed to solve the multimodal FL problem in a paper [77]. The traditional FL model is unable to handle this problem.…”

mentioning

confidence: 99%

Role of federated learning in healthcare systems: A survey

Rana¹,

Marwaha²

2024

MFC

View full text Add to dashboard Cite

show abstract

“…Currently, some studies on the multi-model in AQI prediction are found in [88][89][90], but these applications are implemented on a single machine. Several authors have studied multi-model FL for other fields, as shown in [91][92][93] and [86,94,95]. In this section, some of them are presented with the hope that they will help convey new directions for AQI forecasting in the future.…”

Section: Multi-models Federated Learningmentioning

confidence: 99%

Insights into Multi-Model Federated Learning: An Advanced Approach for Air Quality Index Forecasting

Khoa

Dao

et al. 2022

Algorithms

View full text Add to dashboard Cite

The air quality index (AQI) forecast in big cities is an exciting study area in smart cities and healthcare on the Internet of Things. In recent years, a large number of empirical, academic, and review papers using machine learning (ML) for air quality analysis have been published. However, most of those studies focused on traditional centralized processing on a single machine, and there had been few surveys of federated learning (FL) in this field. This overview aims to fill this gap and provide newcomers with a broader perspective to inform future research on this topic, especially for the multi-model approach. In this survey, we went over the works that previous scholars have conducted in AQI forecast both in traditional ML approaches and FL mechanisms. Our objective is to comprehend previous research on AQI prediction including methods, models, data sources, achievements, challenges, and solutions applied in the past. We also convey a new path of using multi-model FL, which has piqued the computer science community’s interest recently.

show abstract

“…In congruent MFL , the clients hold similar or the same local modality combinations, and horizontal FL is the typical setting of this type. The majority of existing MFL work [ 9 , 10 , 11 , 12 ] has also focused on this federated setting, where all the clients hold the same input modality categories and feature space but differ as to the sample space. In [ 10 ], the authors proposed a multimodal federated learning framework for multimodal activity recognition with an early fusion approach via local co-attention.…”

Section: Introductionmentioning

confidence: 99%

“…The majority of existing MFL work [ 9 , 10 , 11 , 12 ] has also focused on this federated setting, where all the clients hold the same input modality categories and feature space but differ as to the sample space. In [ 10 ], the authors proposed a multimodal federated learning framework for multimodal activity recognition with an early fusion approach via local co-attention. The authors in [ 12 ] provided a detailed analysis of the convergence problem of MFL with late fusion methods under the non-IID setting.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Federated Learning: A Survey

Che

Wang

Zhou³

et al. 2023

Sensors

View full text Add to dashboard Cite

Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.

show abstract

A unified framework for multi-modal federated learning

Cited by 38 publications

References 20 publications

Role of federated learning in healthcare systems: A survey

Role of federated learning in healthcare systems: A survey

Insights into Multi-Model Federated Learning: An Advanced Approach for Air Quality Index Forecasting

Multimodal Federated Learning: A Survey

Contact Info

Product

Resources

About