Towards Efficient and Privacy-Preserving Federated Deep Learning

Meng, Hao; Li, Hongwei; Xu, Guowen; Liu, Sen; Yang, Haomiao

doi:10.1109/icc.2019.8761267

Cited by 138 publications

(71 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Hao et al [57] reported that symmetric additively homomorphic encryption has excellent efficiency and could address computational and communication overhead faced by other public-key encryption schemes.…”

Section: Discussion and Learned Lessonsmentioning

confidence: 99%

“…A recent solution based on encryption was proposed by Hao et al [57]. The process is simple, users perform local training over their private datasets, perturb and encrypt the resulting local gradients, then upload them to the cloud which computes the global encrypted gradients.…”

Section: A2) Server-assisted Collaborative Pp Model Learningmentioning

confidence: 99%

“…The above process is repeated until loss function reaches minimum. The authors of [57] presented a theoretical security analysis and considered the solution as secure against server cloud and compromised users. Besides, performance evaluations described by authors showed better results in terms of communication overhead and computing cost compared to [27], and high accuracy compared to a non-private model.…”

Section: A2) Server-assisted Collaborative Pp Model Learningmentioning

confidence: 99%

See 2 more Smart Citations

A review of privacy-preserving techniques for deep learning

2020

View full text Add to dashboard Cite

Deep learning is one of the advanced approaches of machine learning, and has attracted a growing attention in the recent years. It is used nowadays in different domains and applications such as pattern recognition, medical prediction, and speech recognition. Differently from traditional learning algorithms, deep learning can overcome the dependency on hand-designed features. Deep learning experience is particularly improved by leveraging powerful infrastructures such as clouds and adopting collaborative learning for model training. However, this comes at the expense of privacy, especially when sensitive data are processed during the training and the prediction phases, as well as when training model is shared. In this paper, we provide a review of the existing privacy-preserving deep learning techniques, and propose a novel multilevel taxonomy, which categorizes the current state-of-the-art privacy-preserving deep learning techniques on the basis of privacy-preserving tasks at the top level, and key technological concepts at the base level. This survey further summarizes evaluation results of the reviewed solutions with respect to defined performance metrics. In addition, it derives a set of learned lessons from each privacy-preserving task. Finally, it highlights open research challenges and provides some recommendations as future research directions.

show abstract

Section: Discussion and Learned Lessonsmentioning

confidence: 99%

Section: A2) Server-assisted Collaborative Pp Model Learningmentioning

confidence: 99%

Section: A2) Server-assisted Collaborative Pp Model Learningmentioning

confidence: 99%

See 1 more Smart Citation

A review of privacy-preserving techniques for deep learning

2020

View full text Add to dashboard Cite

show abstract

“…A neural network is an excellent less-computationally sophisticated alternative to many systems that experience high computational overhead. The development in neural network research enables us to use more than one hidden layer learning by a network with more than one hidden layer is called deep learning, which can process a bulk amount of data and is suitable for big data (Hao et al, 2019). Recently, deep learning has been extensively applied in all areas of science and engineering for efficient handling of big data and substantial computational overhead.…”

Section: Privacy-preserving Methods For Big Datamentioning

confidence: 99%

Privacy preserving big data analytics: A critical analysis of state‐of‐the‐art

Pramanik

Lau

Hossain

et al. 2020

WIREs Data Min & Knowl

View full text Add to dashboard Cite

In the era of “big data,” a huge number of people, devices, and sensors are connected via digital networks and the cross‐plays among these entities generate enormous valuable data that facilitate organizations to innovate and grow. However, the data deluge also raises serious privacy concerns which may cause a regulatory backlash and hinder further organizational innovation. To address the challenge of information privacy, researchers have explored privacy‐preserving methodologies in the past two decades. However, a thorough study of privacy preserving big data analytics is missing in existing literature. The main contributions of this article include a systematic evaluation of various privacy preservation approaches and a critical analysis of the state‐of‐the‐art privacy preserving big data analytics methodologies. More specifically, we propose a four‐dimensional framework for analyzing and designing the next generation of privacy preserving big data analytics approaches. Besides, we contribute to pinpoint the potential opportunities and challenges of applying privacy preserving big data analytics to business settings. We provide five recommendations of effectively applying privacy‐preserving big data analytics to businesses. To the best of our knowledge, this is the first systematic study about state‐of‐the‐art in privacy‐preserving big data analytics. The managerial implication of our study is that organizations can apply the results of our critical analysis to strengthen their strategic deployment of big data analytics in business settings, and hence to better leverage big data for sustainable organizational innovation and growth. This article is categorized under: Commercial, Legal, and Ethical Issues > Security and Privacy Fundamental Concepts of Data and Knowledge > Big Data Mining Fundamental Concepts of Data and Knowledge > Data Concepts

show abstract

“…Several communication efficient and privacy preservation distributed approaches have been proposed recently including Practical Secure Aggregation (PSA) [41], Federated Extreme Boosting (XGB) [42], Efficient and Privacy-Preserving Federated Deep Learning (EPFDL) [43] and Privacy-preserving collaborative learning (PPCL) [44] as listed in Table 3. Specifically, PSA and XGB utilise collaborative training in order to resist collusion among adversaries, but both approaches do not guarantee communication efficiency.…”

Section: Functional Comparisonmentioning

confidence: 99%

FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning

2020

View full text Add to dashboard Cite

Artificial Intelligence (AI) has been applied to solve various challenges of real-world problems in recent years. However, the emergence of new AI technologies has brought several problems, especially with regard to communication efficiency, security threats and privacy violations. Towards this end, Federated Learning (FL) has received widespread attention due to its ability to facilitate the collaborative training of local learning models without compromising the privacy of data. However, recent studies have shown that FL still consumes considerable amounts of communication resources. These communication resources are vital for updating the learning models. In addition, the privacy of data could still be compromised once sharing the parameters of the local learning models in order to update the global model. Towards this end, we propose a new approach, namely, Federated Optimisation (FedOpt) in order to promote communication efficiency and privacy preservation in FL. In order to implement FedOpt, we design a novel compression algorithm, namely, Sparse Compression Algorithm (SCA) for efficient communication, and then integrate the additively homomorphic encryption with differential privacy to prevent data from being leaked. Thus, the proposed FedOpt smoothly trade-offs communication efficiency and privacy preservation in order to adopt the learning task. The experimental results demonstrate that FedOpt outperforms the state-of-the-art FL approaches. In particular, we consider three different evaluation criteria; model accuracy, communication efficiency and computation overhead. Then, we compare the proposed FedOpt with the baseline configurations and the state-of-the-art approaches, i.e., Federated Averaging (FedAvg) and the paillier-encryption based privacy-preserving deep learning (PPDL) on all these three evaluation criteria. The experimental results show that FedOpt is able to converge within fewer training epochs and a smaller privacy budget.

show abstract

Towards Efficient and Privacy-Preserving Federated Deep Learning

Cited by 138 publications

References 21 publications

A review of privacy-preserving techniques for deep learning

A review of privacy-preserving techniques for deep learning

Privacy preserving big data analytics: A critical analysis of state‐of‐the‐art

FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning

Contact Info

Product

Resources

About