Differentially Private Aircomp Federated Learning with Power Adaptation Harnessing Receiver Noise

Koda, Yusuke; Yamamoto, Koji; Nishio, Takayuki; Morikura, Masahiro

doi:10.1109/globecom42002.2020.9322199

Cited by 46 publications

(35 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies have investigated different training aspects including personalization (i.e., multi-task learning) [53], robustness guarantees [54], [55], and training over dynamic topologies [56]. To further improve the data privacy against the attacks inverting model parameters into raw data [57], [58], various privacy-preserving methods have been investigated, such as injecting fine-tuned noise into model parameters via a differential privacy mechanism [31], [59]- [61] and mixing model parameters over the air via analog transmissions [62], [63]. Still, one critical issue of FL is that its communication overhead is proportional to the number of model parameters.…”

Section: Federated Learning (Fl)mentioning

confidence: 99%

See 1 more Smart Citation

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

et al. 2021

View full text Add to dashboard Cite

Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. By imbuing intelligence into the network edge, edge nodes can proactively carry out decision-making, and thereby react to local environmental changes and disturbances while experiencing zero communication latency. To achieve this goal, it is essential to cater for high ML inference accuracy at scale under time-varying channel and network dynamics, by continuously exchanging fresh data and ML model updates in a distributed way. Taming this new kind of data traffic boils down to improving the communication efficiency of distributed learning by optimizing communication payload types, transmission techniques, and scheduling, as well as ML architectures, algorithms, and data processing methods. To this end, this article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases. SIGNIFICANCE AND MOTIVATIONThe pursuit of extremely stringent latency and reliability guarantees is essential in the fifth generation (5G) communication system and beyond [1], [2]. In a wirelessly automated factory, the remote control of assembly robots should provision the same level of target latency and reliability offered by existing wired factory systems. To this end, for instance, control packets should be delivered within 1 ms with 99.99999% reliability [3]- [5]. Things are becoming even more challenging in the emerging mission-critical applications beyond 5G. A prime example is the forthcoming nonterrestrial networks consisting of a massive constellation of low-altitude earth orbit (LEO) satellites [6]- [11]. Given such

show abstract

Section: Federated Learning (Fl)mentioning

confidence: 99%

“…In this regard, a joint design of SL (Sec. 5.1) that is robust against non-IID data distributions [82], [194] and feature interpolation and averaging via the Mixup data augmentation (Sec. 5.3) with heterogeneous FoVs and frame rates improving energy efficiency is considered.…”

Section: Heteromodal Sl For Mmwave Channel Predictionmentioning

confidence: 99%

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

et al. 2021

View full text Add to dashboard Cite

show abstract

“…While FL is designed for training over homogeneous agents with a common objective, recent studies have extended the focus towards personalization (i.e., multitask learning) [25], training over dynamic topologies [26] and robustness guarantees [27], [28]. In terms of improving data privacy against malicious attackers, various privacy-preserving methods including injecting fine-tuned noise into model parameters via a differential privacy mechanism [29]- [32] and mixing model parameters over the air via analog transmissions [33], [34] have been recently investigated. Despite of the advancements in FL design, one main drawback in the design of FL is that its communication overhead is proportional to the number of model parameters calling for the design of communication-efficient FL.…”

Section: B Distributed Learning Over Wireless Networkmentioning

confidence: 99%

Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs

Soret

Nguyen

Seeger

et al. 2022

IEEE Trans. on Green Commun. Netw.

View full text Add to dashboard Cite

An Intelligent IoT Environment (iIoTe) is comprised of heterogeneous devices that can collaboratively execute semiautonomous IoT applications, examples of which include highly automated manufacturing cells or autonomously interacting harvesting machines. Energy efficiency is key in such edge environments, since they are often based on an infrastructure that consists of wireless and battery-run devices, e.g., e-tractors, drones, Automated Guided Vehicle (AGV)s and robots. The total energy consumption draws contributions from multiple iIoTe technologies that enable edge computing and communication, distributed learning, as well as distributed ledgers and smart contracts. This paper provides a state-of-the-art overview of these technologies and illustrates their functionality and performance, with special attention to the tradeoff among resources, latency, privacy and energy consumption. Finally, the paper provides a vision for integrating these enabling technologies in energyefficient iIoTe and a roadmap to address the open research challenges.

show abstract

“…and also in medical domain (Rajendran et al, 2021;Kerkouche et al, 2021;Choudhury et al, 2019;Ge et al, 2020). Researchers also investigated the scope of the differentially private algorithm in several applications (Zhao et al, 2020;Koda et al, 2020;Hu et al, 2020;Chen et al, 2018). However, for sequence tagging tasks, the applicability of the FL framework along with differential privacy (DP) is yet to be explored.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the Third Workshop on Privacy in Natural Language Processing

2021

View full text Add to dashboard Cite

Recent works have shown that language models (LMs), e.g., for next word prediction (NWP), have a tendency to memorize rare or unique sequences in the training data. Since useful LMs are often trained on sensitive data, it is critical to identify and mitigate such unintended memorization. Federated Learning (FL) has emerged as a novel framework for large-scale distributed learning tasks. It differs in many aspects from the well-studied central learning setting where all the data is stored at the central server, and minibatch stochastic gradient descent is used to conduct training. This work is motivated by our observation that NWP models trained under FL exhibited remarkably less propensity to such memorization compared to the central learning setting. Thus, we initiate a formal study to understand the effect of different components of FL on unintended memorization in trained NWP models. Our results show that several differing components of FL play an important role in reducing unintended memorization. First, we discover that the clustering of data according to users-which happens by design in FLhas the most significant effect in reducing such memorization. Using the Federated Averaging optimizer with larger effective minibatch sizes for training causes a further reduction. We also demonstrate that training in FL with a userlevel differential privacy guarantee results in models that can provide high utility while being resilient to memorizing out-of-distribution phrases with thousands of insertions across over a hundred users in the training set.(a) A user selected as a secret sharer for a canary. (b) An example in a secret sharer's local dataset being replaced by the canary.

show abstract

Differentially Private Aircomp Federated Learning with Power Adaptation Harnessing Receiver Noise

Cited by 46 publications

References 11 publications

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs

Proceedings of the Third Workshop on Privacy in Natural Language Processing

Contact Info

Product

Resources

About