Privacy-Preserving Distributed Multi-Task Learning with Asynchronous Updates

Xie, Liyang; Baytaş, İnci M.; Lin, Kwei-Jay; Zhou, Jiayu

doi:10.1145/3097983.3098152

Cited by 73 publications

(16 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, while avoiding exposing users' raw data during the learning process, the model updates transmitted in ClusterFL may still reveal certain information about user activities [27,56]. In the future, we will study how to integrate privacy-preserving techniques [55] and investigate the trade-off between privacy and utility [3,53]. Second, we will extend ClusterFL to a wider range of applications, where the nodes' data exhibits intrinsic similarity and locality.…”

Section: Discussionmentioning

confidence: 99%

ClusterFL

Ouyang

Xie

Zhou

et al. 2021

Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services

Self Cite

111

View full text Add to dashboard Cite

Federated Learning (FL) has recently received significant interests thanks to its capability of protecting data privacy. However, existing FL paradigms yield unsatisfactory performance for a wide class of human activity recognition (HAR) applications since they are oblivious to the intrinsic relationship between data of different users. We propose ClusterFL, a similarity-aware federated learning system that can provide high model accuracy and low communication overhead for HAR applications. ClusterFL features a novel clustered multi-task federated learning framework that maximizes the training accuracy of multiple learned models while automatically capturing the intrinsic clustering relationship among the data of different nodes. Based on the learned cluster relationship, ClusterFL can efficiently drop out the nodes that converge slower or have little correlation with other nodes in each cluster, significantly speeding up the convergence while maintaining the accuracy performance. We evaluate the performance of ClusterFL on an NVIDIA edge testbed using four new HAR datasets collected from total 145 users. The results show that, ClusterFL outperforms several state-of-the-art FL paradigms in terms of overall accuracy, and save more than 50% communication overhead at the expense of negligible accuracy degradation.

show abstract

Section: Discussionmentioning

confidence: 99%

ClusterFL

Ouyang

Xie

Zhou

et al. 2021

Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services

Self Cite

111

View full text Add to dashboard Cite

show abstract

“…In addition to introducing differential privacy into standalone machine learning, differentially private transfer learning has also been investigated [133], [134]. Transfer learning aims to transfer knowledge from source domains to improve learning performance in target domains [134].…”

Section: Future Research Directions 61 Private Transfer Learningmentioning

confidence: 99%

More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence

Zhu¹,

Ye²,

Wang³

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just privacy preservation. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

show abstract

“…They also developed an attribute-wise noise addition scheme in their algorithm. Xie et al [26] proposed a distributed multitask framework for privacy preservation to preserve sensitive data and private information that may be contained in the distributed data. The proposed method is a privacy-preserving proximal gradient algorithm which asynchronously updates models of the learning tasks and solves a general class of multi-task learning formulations.…”

Section: B Privacy-preserving Recommentation Systemmentioning

confidence: 99%

A Privacy-Preserving Multi-Task Framework for Knowledge Graph Enhanced Recommendation

Zhou

Zhang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Multi-task learning (MTL) is a learning paradigm which can improve generalization performance by transferring knowledge among multiple tasks. Traditional collaborative filtering recommendation methods suffer from cold start, sparsity and scalability problems. The latest research has shown that applying side information of knowledge graph can not only solve the problems above, but also improve the accuracy of recommendation. However, existing multi-task methods for knowledge graph enhanced recommendation expose obvious issues of disclosing the private information of training samples. In order to solve these problems, we put forward a privacy-preserving multi-task framework for knowledge graph enhanced recommendation. In specific, Laplacian noise is added into the recommendation module to guarantee the privacy of sensitive data and knowledge graph is utilized to improve the accuracy of recommendation. Extensive experimental results on three datasets demonstrate that the proposed method can not only preserve the privacy of sensitive training data, but also have little effect on the prediction accuracy of the model.

show abstract

Privacy-Preserving Distributed Multi-Task Learning with Asynchronous Updates

Cited by 73 publications

References 23 publications

ClusterFL

ClusterFL

More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence

A Privacy-Preserving Multi-Task Framework for Knowledge Graph Enhanced Recommendation

Contact Info

Product

Resources

About