Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Guliani, Dhruv; Beaufays, Françoise; Motta, Giovanni

doi:10.1109/icassp39728.2021.9413397

Cited by 56 publications

(32 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These might be better at preserving this internal language model, as they freeze more of the original model weights. Finally, smaller ASR model size and federated learning-although early results have noted difficulties in applying it to ASR [57], effort is underway to lower the training cost and improve accuracy [15]-might bring about the potential for ASR individualization to be targeted at the level of ideolects, with people able to use ASR model's tailored to their personal accent profile.…”

Section: Discussionmentioning

confidence: 99%

Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding

Sullivan¹,

Shibano²,

Abdul-Mageed³

2022

Preprint

View full text Add to dashboard Cite

ASR systems designed for native English (L1) usually underperform on non-native English (L2). To address this performance gap, (i) we extend our previous work to investigate fine-tuning of a pre-trained wav2vec 2.0 model [2,56] under a rich set of L1 and L2 training conditions. We further (ii) incorporate language model decoding in the ASR system, along with the fine-tuning method. Quantifying gains acquired from each of these two approaches separately and an error analysis allows us to identify different sources of improvement within our models. We find that while the large self-trained wav2vec 2.0 may be internalizing sufficient decoding knowledge for clean L1 speech [56], this does not hold for L2 speech and accounts for the utility of employing language model decoding on L2 data.

show abstract

Section: Discussionmentioning

confidence: 99%

Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding

Sullivan¹,

Shibano²,

Abdul-Mageed³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Federated learning was first utilized to enhance Google's Android keyboard prediction [5] without submitting the user's credentials data to the cloud in IoT applications. Apple also employs federated learning to improve Siri's voice recognition [6]. Undoubtedly, blockchain technology has used federated learning to adjust the model and preserve the organization's privacy and data.…”

Section: Introductionmentioning

confidence: 99%

Centralized Machine Learning Versus Federated Averaging: A Comparison using MNIST Dataset

2022

KSII TIIS

View full text Add to dashboard Cite

A flood of information has occurred with the rise of the internet and digital devices in the fourth industrial revolution era. Every millisecond, massive amounts of structured and unstructured data are generated; smartphones, wearable devices, sensors, and self-driving cars are just a few examples of devices that currently generate massive amounts of data in our daily. Machine learning has been considered an approach to support and recognize patterns in data in many areas to provide a convenient way to other sectors, including the healthcare sector, government sector, banks, military sector, and more. However, the conventional machine learning model requires the data owner to upload their information to train the model in one central location to perform the model training. This classical model has caused data owners to worry about the risks of transferring private information because traditional machine learning is required to push their data to the cloud to process the model training. Furthermore, the training of machine learning and deep learning models requires massive computing resources. Thus, many researchers have jumped to a new model known as "Federated Learning". Federated learning is emerging to train Artificial Intelligence models over distributed clients, and it provides secure privacy information to the data owner. Hence, this paper implements Federated Averaging with a Deep Neural Network to classify the handwriting image and protect the sensitive data. Moreover, we compare the centralized machine learning model with federated averaging. The result shows the centralized machine learning model outperforms federated learning in terms of accuracy, but this classical model produces another risk, like privacy concern, due to the data being stored in the data center. The MNIST dataset was used in this experiment.

show abstract

“…As strict regulations emerge for data capture and storage such as GDPR [23], CCPA [77], distributed deep learning is being used to enable privacy-aware personalization across a wide range of web clients and smart edge devices with varying resource constraints. For instance, distributed deep learning is replacing third-party cookies in the chrome browser for ad-personalization [10,20], enabling next-word prediction on mobile devices [29], speaker verification on smart home assistants [26], HIPPA-compliant diagnosis on clinical devices [68] and real-time navigation in vehicles [19].…”

Section: Introductionmentioning

confidence: 99%

AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning

Chopra¹,

Sahu²,

Singh³

et al. 2021

Preprint

View full text Add to dashboard Cite

Distributed deep learning frameworks like federated learning (FL) and its variants are enabling personalized experiences across a wide range of web clients and mobile/IoT devices. However, these FLbased frameworks are constrained by computational resources at clients due to the exploding growth of model parameters (eg. billion parameter model). Split learning (SL), a recent framework, reduces client compute load by splitting the model training between client and server. This flexibility is extremely useful for low-compute setups but is often achieved at cost of increase in bandwidth consumption and may result in sub-optimal convergence, especially when client data is heterogeneous. In this work, we introduce AdaSplit which enables efficiently scaling SL to low resource scenarios by reducing bandwidth consumption and improving performance across heterogeneous clients. To capture and benchmark this multidimensional nature of distributed deep learning, we also introduce C3-Score, a metric to evaluate performance under resource budgets. We validate the effectiveness of AdaSplit under limited resources through extensive experimental comparison with strong federated and split learning baselines. We also present a sensitivity analysis of key design choices in AdaSplit which validates the ability of AdaSplit to provide adaptive trade-offs across variable resource budgets.

show abstract

Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework

Cited by 56 publications

References 11 publications

Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding

Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding

Centralized Machine Learning Versus Federated Averaging: A Comparison using MNIST Dataset

AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning

Contact Info

Product

Resources

About