Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures

Abstract

Recent publications on automatic-speech-recognition (ASR) have a strong focus on attention encoder-decoder (AED) architectures which work well for large datasets, but tend to overfit when applied in low resource scenarios. One solution to tackle this issue is to generate synthetic data with a trained text-tospeech system (TTS) if additional text is available. This was successfully applied in many publications with AED systems. We present a novel approach of silence correction in the data pre-processing for TTS systems which increases the robustness when training on corpora targeted for ASR applications. In this work we do not only show the successful application of synthetic data for AED systems, but also test the same method on a highly optimized state-of-the-art Hybrid ASR system and a competitive monophone based system using connectionisttemporal-classification (CTC). We show that for the later systems the addition of synthetic data only has a minor effect, but they still outperform the AED systems by a large margin on LibriSpeech-100h. We achieve a final word-error-rate of 3.3%/10.0% with a Hybrid system on the clean/noisy test-sets, surpassing any previous state-of-the-art systems that do not include unlabeled audio data.

End-to-End Speech Recognition: A Survey

“…SpecAugment also is shown to improve classical hybrid HMM models [297]. TTS on the other hand considerably improved attention-based encoder-decoder models trained on limited resources, but did not reach the performance of other E2E approaches or hybrid HMM models, which in turn were not considerably improved by TTS [179]. Multilingual approaches also help improve ASR development for low resource tasks, again both for classical [298], as well as for E2E systems [299], [300].…”

“…Most data augmentation methods perform data perturbation by exploiting certain dimensions of speech signal variation: speed perturbation [172], [173], vocal tract length perturbation [174], [172], frequency axis distortion [172], sequence noise injection [175], SpecAugment [176], or semantic mask [177]. Also, text-only data may be used to generate data using textto-speech (TTS) on feature [178] or signal level [179]. In a comparison of the effect of TTS-based data augmentation on different E2E ASR architectures in [179], AED seemed to be the only architecture that took advantage of data generated by TTS, significantly.…”

“…Also, text-only data may be used to generate data using textto-speech (TTS) on feature [178] or signal level [179]. In a comparison of the effect of TTS-based data augmentation on different E2E ASR architectures in [179], AED seemed to be the only architecture that took advantage of data generated by TTS, significantly.…”

End-to-End Speech Recognition: A Survey

“…Most data augmentation methods perform data perturbation by exploiting certain dimensions of speech signal variation: speed perturbation [201], [202], vocal tract length perturbation [201], [203], frequency axis distortion [201], sequence noise injection [204], SpecAugment [205], or semantic mask [206]. Also, text-only data may be used to generate data using textto-speech (TTS) on feature [207] or signal level [208]. In a comparison of the effect of TTS-based data augmentation on different E2E ASR architectures in [208], AED seemed to be the only architecture that appeared to benefit significantly from the TTS data.…”

“…Also, text-only data may be used to generate data using textto-speech (TTS) on feature [207] or signal level [208]. In a comparison of the effect of TTS-based data augmentation on different E2E ASR architectures in [208], AED seemed to be the only architecture that appeared to benefit significantly from the TTS data.…”