Personalized Federated Learning With Differential Privacy

On Consensus-Optimality Trade-offs in Collaborative Deep Learning

“…Scaling up deep learning algorithms in a distributed setting ( Recht et al, 2011 ; LeCun et al, 2015 ; Jin et al, 2016 ) is becoming increasingly critical, impacting several applications such as learning in robotic networks ( Lenz et al, 2015 ; Fang et al, 2019 ), the Internet of Things (IoT) ( Gubbi et al, 2013 ; Lane et al, 2015 ; Hu et al, 2020 ), mobile device networks ( Lane and Georgiev, 2015 ; Kang et al, 2020 ), and sensor networks ( Ge et al, 2019 ; He et al, 2020 ). For instance, with the development of wireless communication and distributed computing technologies, intelligent sensor network has been emerging as a kind of large-scale distributed network systems, which request more advanced sensor fusion techniques that enable data privacy preservation ( Jiang et al, 2017a ; He et al, 2019 ), dynamic optimization ( Yang et al, 2016 ), and intelligent learning ( Tan, 2020 ).…”

Federated Learning for Internet of Things: A Comprehensive Survey

“…• Differential privacy is applied in [189], [190] by adding artificial noise (e.g., Gaussian noise) to learning gradients to preserve training data and hidden personal information against external threats. • A secure aggregation scheme is proposed in [191] for safe FL systems, in order to provide the strongest possible security for data aggregation.…”

“…Perturbation techniques [188] such as differential privacy or dummy can be used to protect training datasets against data breach, by constructing composition theorems with complex mathematical solutions. As an example, differential privacy is applied in [189] by inserting artificial noise (e.g., Gaussian noise) to the gradients of neural network layers to preserve training data and hidden personal information against external threats while the convergence property is guaranteed. This solution would ensure that the server or malicious users cannot learn much additional information of user samples from the received messages under any auxiliary information and attack.…”

Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization

“…To address this issue, several privacy-preserving framework have been proposed, among which differential privacy (DP) [Dwork and Roth, 2014] has become the de-facto standard due to its rigorous privacy guarantee and effectiveness in data analysis tasks [Abadi et al, 2016;Hu et al, 2020;Huang et al, 2019;Guo and Gong, 2018;Gong et al, 2016]. General DP mechanisms, such as Gaussian or Laplacian mechanism, rely on the injection of carefully calibrated noise to the output of an algorithm directly.…”