DeceFL: a principled fully decentralized federated learning framework

Abstract

Traditional machine learning relies on a centralized data pipeline for model training in various applications; however, data are inherently fragmented. Such a decentralized nature of databases presents the serious challenge for collaboration: sending all decentralized datasets to a central server raises serious privacy concerns. Although there has been a joint effort in tackling such a critical issue by proposing privacy-preserving machine learning frameworks, such as federated learning, most state-of-the-art frameworks are built still in a centralized way, in which a central client is needed for collecting and distributing model information (instead of data itself) from every other client, leading to high communication burden and high vulnerability when there exists a failure at or an attack on the central client. Here we propose a principled decentralized federated learning algorithm (DeceFL), which does not require a central client and relies only on local information transmission between clients and their neighbors, representing a fully decentralized learning framework. It has been further proven that every client reaches the global minimum with zero performance gap and achieves the same convergence rate O(1/T ) (where T is the number of iterations in gradient descent) as centralized federated learning when the loss function is smooth and strongly convex. Finally, the proposed algorithm has been applied to a number of applications to illustrate its effectiveness for both convex and nonconvex loss functions, time-invariant and time-varying topologies, as well as IID and Non-IID of datasets, demonstrating its applicability to a wide range of real-world medical and industrial applications.

Existing Barriers Faced by and Future Design Recommendations for Direct-to-Consumer Health Care Artificial Intelligence Apps: Scoping Review

“…Privacy protection has become a hot topic in the health care AI research field [ 74 ], with numerous studies dedicated to developing innovative privacy-preserving solutions without compromising the performance of big data–driven AI models. These include developing privacy-enhancing technologies, such as homomorphic encryption [ 75 ], securing multiparty computation and differential privacy [ 76 ], and exploring new training methods and data governance models, such as distributed federated machine learning using synthesized data from multiple organizations [ 77 ], data-sharing pools [ 78 ], data trusts [ 79 ], and data cooperatives [ 80 ]. Second, the lack of clarity in accountability and regulation has also been universally identified in prior research as a key obstacle to the application of AI in health care [ 81 - 83 ].…”

Semi-supervised small-sample gearbox fault diagnosis with privacy protection

“…Chen 等人 [19] 提出了一种基于差分加权联邦平均的联 邦迁移学习框架，协同训练诊断模型。Maet [20] 在联 邦学习中采用扩展 Kalman 作为模型更新算法来解决 流量预测中潜在的攻击。然而，这些方法在隐私保护 过程中严重依赖于中央服务器。此外，由于数据加密 而导致的全局模型精度下降以及计算和通信开销的 增加等问题也不容忽视。 Warnat [21] 提出了一种群学习 方法， 该方法在每次迭代中根据特定的规则选择一个 变化的领导者，从而实现去中心化。然而，该网络只 是转移了中央服务器，而没有实现完全的去中心化。 同时，该框架没有解决网络的异构问题。为了解决上 述问题，本文采用去中心化联邦学习 [22] DeceFL，利…”