Networked Federated Multi-Task Learning

SarcheshmehPour, Yasmin; Tian, Yu; Zhang, Linli; Jung, Alexander

doi:10.36227/techrxiv.14685696.v1

Cited by 3 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By considering each FL client as a task in MTL, the server can learn and discern the relationships among clients based on their heterogeneous local data. MOCHA [10] expands distributed MTL to the FL context, learning personalized models for each client; however, all clients must participate in each round of FL model training. MOCHA is only applicable to convex models and has limited generalization in deep learning.…”

Section: Related Workmentioning

confidence: 99%

“…To enhance the accuracy of FL in data-heterogeneous scenarios, researchers have introduced various similarity-based personalized federated learning (PFL) approaches [9], including multi-task learning (MTL) [10][11][12], model interpolation [13][14][15], and clustering [16][17], aiming to mitigate the weight divergence issue. However, most methods addressing non-IID FL present considerable drawbacks, as they lead to significant computation and communication overhead.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PFL-LDG: Privacy-preserving Federated Learning via Lightweight Device Grouping

Wang,

Liu,

Liu

2023

2023 IEEE Smart World Congress (SWC)

View full text Add to dashboard Cite

The rapid growth of private data from distributed edge networks, driven by the proliferation of IoT sensors, wearable devices, and smartphones, offers significant opportunities for AI applications. However, traditional distributed machine learning methods struggle to address data privacy concerns effectively. Federated learning (FL) has appeared as a popular, innovative paradigm for distributed machine learning that enables collaborative training of models across multiple data silos while preserving privacy. Yet, in large-scale and complex edge networks, the convergence performance of existing FL methods deteriorates when dealing with highly heterogeneous data. This paper introduces PFL-LDG, a similarity-based lightweight privacy-protected grouping FL method that mitigates the impact of non-IID data on FL model performance in dataheterogeneous scenarios. Unlike conventional FL, PFL-LDG clusters devices based on data distribution similarity, reducing inefficiency and straggler issues while supplying personalized FL models for edge devices and enhancing FL accuracy. The paper's main contribution is the proposal of a novel similaritybased lightweight privacy-protected grouping FL framework, focusing on improving privacy protection and training efficiency in heterogeneous edge resource-constrained FL systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

PFL-LDG: Privacy-preserving Federated Learning via Lightweight Device Grouping

Wang,

Liu,

Liu

2023

2023 IEEE Smart World Congress (SWC)

View full text Add to dashboard Cite

show abstract

“…As a future work, one can put this work into context of the regularizated empirical risk minimization and its special case generalised total variation minimization, see [29]- [30]. Moreover, one can put this work into context a line of work total variation minimisation methods Motivated by a clustered signal; see [31]- [33], which, in contrast to our approach, use a non-smooth variant of total variation.…”

Section: Adaptive Algorithmmentioning

confidence: 99%

Robust Adaptive Generalized Correntropy-based Smoothed Graph Signal Recovery with a Kernel Width Learning

Torkamani¹,

Zayyani²,

Marvasti³

2022

Preprint

View full text Add to dashboard Cite

This paper proposes a robust adaptive algorithm for smooth graph signal recovery which is based on generalized correntropy. A proper cost function is defined, which takes the smoothness and generalized correntropy into account. The generalized correntropy used in this paper employs the generalized Gaussian density (GGD) function as the kernel. The proposed adaptive algorithm is derived and a kernel width learning-based version of the algorithm is suggested. The simulation results confirm the performance of the proposed algorithm for learning the kernel-width to the fixed correntropy kernel version of the algorithm. Moreover, some theoretical analysis of the proposed algorithm are provided. In this regard, firstly, the convexity analysis of the cost function is discussed. Secondly, the uniform stability of the algorithm is investigated. Thirdly, the mean convergence analysis is also added. Finally, the computational complexity analysis of the algorithm is incorporated. In addition, some synthetic and real-world experiments show the efficiency of the proposed algorithm in comparison to some other adaptive algorithms in the literature of adaptive graph signal recovery.

show abstract