AdaFed: Performance-based Adaptive Federated Learning

Abstract

Federated Learning is a distributed and privacy-preserving machine learning technique that allows local clients to learn a model without sharing their own data by coordinating with a global server. In this work, we present the Adaptive Federated Learning (AdaFed) algorithm, which aims at improving the training performance of deep neural networks in Federated Learning settings by: (i) dynamically weighting the local models in the model averaging procedure; (ii) by adapting the loss function used by the federation at every communication round. We discuss the specialisation of AdaFed for both classification and regression tasks, providing several validation examples. Due to its adaptive design, the AdaFed algorithm showed a robust behaviour against unbalanced data distributions and adversarial clients.

CADUCEO: A Platform to Support Federated Healthcare Facilities through Artificial Intelligence

“…In addition to FL solutions developed ad hoc for the particular application under consideration, such as the previous ones, theoretical solutions have been proposed in the last period, aimed at enhancing FedAvg. Among them, FedProx [85] allows devices with optimization characteristics strongly different from others to be included in the optimization process, AdaFed [86] replaces the averaging mechanism of clients with a dynamic and adaptive heuristic weighting mechanism based on client performance, FOCUS [87] replaces the averaging mechanism with a credibility mechanism based on Shapley values [88], DecFedAvg [89] and CFA-GE [90] represent the first attempts to decentralize FedAvg by allowing each client to communicate with its neighbors, and FedLCon [91], the first consensus-based FL algorithm.…”

Decentralised Federated Learning for Hospital Networks With Application to COVID-19 Detection

“…end for 19: end for 20: set wi (t) = w i (t − 1) 21: return wi (t) to the server We report the pseudo-code for AdaFed (see Algorithm 2) and refer the reader to [2] for a more detailed discussion of the algorithm.…”

“…Figures 4 and 5 show that the two proposed algorithms exhibit a similar performance across all the communication rounds, with AdaFed converging slightly faster to the final value. we remark that, in federation with balanced and IID data, AdaFed is expected to perform very similarly to FedAvg , as its dynamic weight update has a greater effect on uneven data distributions [2]. For benchmarking purposes, in Figure 4 we also include a dashed line that represents the performance attained after 60 epochs by a single, centralized, server that trains the same neural network on the entirety of the data.…”

“…The original FedAvg [1] algorithm averages the clients models with an a-priori weighting strategy that typically depends on the numerosity of the various clients datasets. On the contrary, the Adaptive Federated Learning (AdaFed) algorithm [2], recently proposed by the authors, envisages a dynamic and adaptive heuristic weighting scheme that takes into account the performance attained by the various clients, so that the impact of low performing and/or malicious clients can be reduced. Both FedAvg and AdaFed share the same overall architecture, with a centralized server orchestrating the learning procedure of the entire federation.…”

ASCFL: Accurate and Speedy Semi-Supervised Clustering Federated Learning

“…With the rapid spread of privatized smart devices [1][2][3][4] , huge amounts of personal privatized data are being generated and stored. In order to protect the clients' privacy, Federated Learning (FL) [5][6][7] is proposed, which can train a global model on the premise of protecting clients' privacy. Recently, FL has been Jingyi He, Biyao Gong, Jiadi Yang, Hai Wang, Pengfei Xu, and Tianzhang Xing are with the School of Information Science and Technology, Northwest University, Xi'an 710100, China.…”