Differentially-private Federated Neural Architecture Search

Singh, Ishika; Zhou, Haoyi; Yang, Kun‐Lin; Ding, Meng; Lin, Bin; Xie, Pengtao

doi:10.48550/arxiv.2006.10559

Cited by 6 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In terms of network topology, a variety of network typologies such as vertical FL [34,35,36,37,38,39,40], split learning [41,42], decentralized FL [43,44,45,46], hierarchical FL [47,48,49,50,51,52], and meta FL [53,54,55] have been proposed. In terms of exchanged information, aside from exchanging gradients and models, recent FL algorithms propose to exchange information such as pseudo labels in semi-supervised FL [56] and architecture parameters in neural architecture search-based FL [5,57,58]. In terms of training procedures, the training procedures in federated GAN [59,60] and transfer learning-based FL [61,62,63,64,65] are significantly different from the vanilla FedAvg algorithm [66].…”

Section: Introductionmentioning

confidence: 99%

FedML: A Research Library and Benchmark for Federated Machine Learning

He,

Li,

et al. 2020

Preprint

105

159

View full text Add to dashboard Cite

Federated learning is a rapidly growing research field in the machine learning domain. Although considerable research efforts have been made, existing libraries cannot adequately support diverse algorithmic development (e.g., diverse topology and flexible message exchange), and inconsistent dataset and model usage in experiments make fair comparisons difficult. In this work, we introduce FedML, an open research library and benchmark that facilitates the development of new federated learning algorithms and fair performance comparisons. FedML supports three computing paradigms (distributed training, mobile on-device training, and standalone simulation) for users to conduct experiments in different system environments. FedML also promotes diverse algorithmic research with flexible and generic API design and reference baseline implementations. A curated and comprehensive benchmark dataset for the non-I.I.D setting aims at making a fair comparison. We believe FedML can provide an efficient and reproducible means of developing and evaluating algorithms for the federated learning research community. We maintain the source code, documents, and user community at https://FedML.ai.

show abstract

Section: Introductionmentioning

confidence: 99%

FedML: A Research Library and Benchmark for Federated Machine Learning

He,

Li,

et al. 2020

Preprint

105

159

View full text Add to dashboard Cite

show abstract

“…However, only a limited amount of research on the influence of noni.i.d. distribution on federated NAS has been reported (He, Annavaram, and Avestimehr 2020;Singh et al 2020). Adversarial training was primarily developed for i.i.d.…”

Section: (D) Optimization For Defense Mechanismsmentioning

confidence: 99%

Federated Learning: Balancing the Thin Line Between Data Intelligence and Privacy

Mathews¹,

Assefa²

2022

Preprint

View full text Add to dashboard Cite

Federated learning holds great promise in learning from fragmented sensitive data and has revolutionized how machine learning models are trained. This article provides a systematic overview and detailed taxonomy of federated learning. We investigate the existing security challenges in federated learning and provide a comprehensive overview of established defense techniques for data poisoning, inference attacks, and model poisoning attacks. The work also presents an overview of current training challenges for federated learning, focusing on handling non-i.i.d. data, high dimensionality issues, heterogeneous architecture, and discusses several solutions for the associated challenges. Finally, we discuss the remaining challenges in managing federated learning training and suggest focused research directions to address the open questions. Potential candidate areas for federated learning, including IoT ecosystem, healthcare applications, are discussed with a particular focus on banking and financial domains. Landscape of Federated Learning 1.1 Need for Federated LearningData has always been of significant priority for businesses of all sizes, especially in Financial Services. With the advancement of technology, companies tend to capture customer data from many sources, such as tracking customer's activities and appending other data sources to proprietary sources to enhance their ability to contextualize data and draw new insights. However, preserving customer data privacy is vital considering the sensitive nature of customer data. (Hahn, Munir, and Mohanty 2018;Yang et al. 2019).Several factors have driven the global consensus to focus on preserving data privacy and security of individuals, businesses, and societies. As a result, governments are strengthening data security and privacy protection measures. For example, General Data Protection Regulations (GDPR) aims to protect the user's privacy by requiring operants to distinctly obtain user consent and adhere to sufficient data privacy requirements (Khan et al. 2019). The establishment of these laws and regulations poses new challenges to the traditional data processing mode of Artificial

show abstract

“…• There is an increasing demand for automated machine learning (AutoML) and lots of algorithms have been presented on neural architecture search (NAS) in practical scenarios [22,133]. However, only a limited amount of research on federated NAS has been reported [145,121,48,169] and little work has considered the influence of Non-IID distributions.…”

Section: Remaining Challenges and Future Directionsmentioning

confidence: 99%

Federated Learning on Non-IID Data: A Survey

Zhu

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we provide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, current research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.

show abstract

Differentially-private Federated Neural Architecture Search

Cited by 6 publications

References 16 publications

FedML: A Research Library and Benchmark for Federated Machine Learning

FedML: A Research Library and Benchmark for Federated Machine Learning

Federated Learning: Balancing the Thin Line Between Data Intelligence and Privacy

Federated Learning on Non-IID Data: A Survey

Contact Info

Product

Resources

About