Federated Acoustic Model Optimization for Automatic Speech Recognition

Tan, Carol; Jiang, Di; Mo, Huaxiao; Peng, Jinhua; Tong, Yongxin; Zhao, Weiwei; Chen, Chaotao; Lian, Rongzhong; Song, Yuanfeng; Xu, Qian

doi:10.1007/978-3-030-59419-0_54

“…Shi et al developed a semantic consistent approach of FL for topic discovery [56]. Tan et al designed a new acoustic model optimization method for FL in the task of automatic speech recognition [58]. Most existing FL solutions aim to learn generic data (e.g., table data, text data).…”

Section: Federated Machine Learningmentioning

confidence: 99%

Federated Dynamic Graph Neural Networks with Secure Aggregation for Video-based Distributed Surveillance

Jiang

¹

,

Jung

²

,

Karl

³

et al. 2022

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Distributed surveillance systems have the ability to detect, track, and snapshot objects moving around in a certain space. The systems generate video data from multiple personal devices or street cameras. Intelligent video-analysis models are needed to learn dynamic representation of the objects for detection and tracking. Can we exploit the structural and dynamic information, without storing the spatiotemporal video data at a central server that leads to a violation of user privacy? In this work, we introduce Federated Dynamic Graph Neural Network (Feddy), a distributed and secured framework to learn the object representations from graph sequences: (1) It aggregates structural information from nearby objects in the current graph as well as dynamic information from those in the previous graph. It uses a self-supervised loss of predicting the trajectories of objects. (2) It is trained in a federated learning manner. The centrally located server sends the model to user devices. Local models on the respective user devices learn and periodically send their learning to the central server without ever exposing the user’s data to server. (3) Studies showed that the aggregated parameters could be inspected though decrypted when broadcast to clients for model synchronizing, after the server performed a weighted average. We design an appropriate aggregation mechanism of secure aggregation primitives that can protect the security and privacy in federated learning with scalability. Experiments on four video camera datasets as well as simulation demonstrate that Feddy achieves great effectiveness and security.

show abstract

“…In [7], various federated averaging schemes and data augmentation techniques have been studied to improve keyword spotting models with data not independent and identically distributed (iid) at the edge. An interactive system was built in [8] to demonstrate how FL can help transfer learning on acoustic models using edge device data. In [9], a federated transfer learning platform is introduced with improved performance using enhanced federated averaging via hierarchical optimization and gradient selection.…”

Section: Introductionmentioning

confidence: 99%

Federated Acoustic Modeling for Automatic Speech Recognition

Cui

¹

,

Kingsbury

²

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Data privacy and protection is a crucial issue for any automatic speech recognition (ASR) service provider when dealing with clients. In this paper, we investigate federated acoustic modeling using data from multiple clients. A client's data is stored on a local data server and the clients communicate only model parameters with a central server, and not their data. The communication happens infrequently to reduce the communication cost. To mitigate the non-iid issue, client adaptive federated training (CAFT) is proposed to canonicalize data across clients. The experiments are carried out on 1,150 hours of speech data from multiple domains. Hybrid LSTM acoustic models are trained via federated learning and their performance is compared to traditional centralized acoustic model training. The experimental results demonstrate the effectiveness of the proposed federated acoustic modeling strategy. We also show that CAFT can further improve the performance of the federated acoustic model.

show abstract

Federated Acoustic Modeling For Automatic Speech Recognition

Cui

¹

,

Kingsbury

²

2021

Preprint

View full text Add to dashboard Cite

Data privacy and protection is a crucial issue for any automatic speech recognition (ASR) service provider when dealing with clients. In this paper, we investigate federated acoustic modeling using data from multiple clients. A client's data is stored on a local data server and the clients communicate only model parameters with a central server, and not their data. The communication happens infrequently to reduce the communication cost. To mitigate the non-iid issue, client adaptive federated training (CAFT) is proposed to canonicalize data across clients. The experiments are carried out on 1,150 hours of speech data from multiple domains. Hybrid LSTM acoustic models are trained via federated learning and their performance is compared to traditional centralized acoustic model training. The experimental results demonstrate the effectiveness of the proposed federated acoustic modeling strategy. We also show that CAFT can further improve the performance of the federated acoustic model.

show abstract

Federated Acoustic Model Optimization for Automatic Speech Recognition

Cited by 3 publications

References 3 publications

Federated Dynamic Graph Neural Networks with Secure Aggregation for Video-based Distributed Surveillance

Federated Dynamic Graph Neural Networks with Secure Aggregation for Video-based Distributed Surveillance

Federated Acoustic Modeling for Automatic Speech Recognition

Federated Acoustic Modeling For Automatic Speech Recognition

Contact Info

Product

Resources

About