REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling

Hu, Hu; Yang, Xuesong; Raeesy, Zeynab; Guo, Jianmin; Keskin, Gokce; Arsikere, Harish; Rastrow, Ariya; Stolcke, Andreas; Maas, Roland

doi:10.48550/arxiv.2012.07353

Cited by 1 publication

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using domain adversarial training, with the discriminator being an accent classifier, has shown significant improvements over standard ASR models (Sun et al, 2018). Pre-training the accent classifier (Das et al, 2021b) and clusteringbased accent relabelling (Hu et al, 2020) have also led to further performance improvements. The use of generative adversarial networks for this task has also been explored (Chen et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Accented Speech Recognition With Accent-specific Codebooks

Prabhu,

Jyothi,

Ganapathy

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Speech accents pose a significant challenge to state-of-the-art automatic speech recognition (ASR) systems. Degradation in performance across underrepresented accents is a severe deterrent to the inclusive adoption of ASR. In this work, we propose a novel accent adaptation approach for end-to-end ASR systems using cross-attention with a trainable set of codebooks. These learnable codebooks capture accent-specific information and are integrated within the ASR encoder layers. The model is trained on accented English speech, while the test data also contained accents which were not seen during training. On the Mozilla Common Voice multi-accented dataset, we show that our proposed approach yields significant performance gains not only on the seen English accents (up to 37% relative improvement in word error rate) but also on the unseen accents (up to 5% relative improvement in WER). Further, we illustrate benefits for a zero-shot transfer setup on the L2Artic dataset. We also compare the performance with other approaches based on accent adversarial training.

show abstract