Data Augmentation Using Prosody and False Starts to Recognize Non-Native Children’s Speech

Abstract

This paper describes AaltoASR's speech recognition system for the INTERSPEECH 2020 shared task on Automatic Speech Recognition (ASR) for non-native children's speech. The task is to recognize non-native speech from children of various age groups given a limited amount of speech. Moreover, the speech being spontaneous has false starts transcribed as partial words, which in the test transcriptions leads to unseen partial words. To cope with these two challenges, we investigate a data augmentation-based approach. Firstly, we apply the prosodybased data augmentation to supplement the audio data. Secondly, we simulate false starts by introducing partial-word noise in the language modeling corpora creating new words. Acoustic models trained on prosody-based augmented data outperform the models using the baseline recipe or the SpecAugment-based augmentation. The partial-word noise also helps to improve the baseline language model. Our ASR system, a combination of these schemes, is placed third in the evaluation period and achieves the word error rate of 18.71%. Post-evaluation period, we observe that increasing the amounts of prosody-based augmented data leads to better performance. Furthermore, removing low-confidence-score words from hypotheses can lead to further gains. These two improvements lower the ASR error rate to 17.99%.

Vowel Non-Vowel Based Spectral Warping and Time Scale Modification for Improvement in Children’s ASR

“…To resolve this issue we investigated two types of time-scale modification techniques. The realtime iterative spectrogram inversion with look-ahead (RTISI-LA) algorithm [23,24,25] constructs a high-quality timedomain signal from its short-time magnitude spectrum with varying parameter s from scale 0.65 to 1.85 with step size 0.10. We also investigated synchronized overlap-add fixed synthesis (SOLAFS) [26] based time scale modification.…”

Synthesis Speech Based Data Augmentation for Low Resource Children ASR

“…We then augment the modified data to the original corpora for further system development. To modify the pitch and speaking rate, we have explored Time Scale Modification (TSM) based on Real-Time Iterative Spectrogram Inversion with Look-Ahead (RTISI-LA) algorithm [16,31]. Both these prosody parameters are tunable and we varied the pitch modification factorr s from 0.65 to 1.45 to modify pitch and the speaking rate modification factor α from 0.65 to 1.85 with a step size of 0.10.…”

“…So far, few general purpose ASR has focused upon data augmentation using the training speakers only. This can be performed, for example, by prosody modification of the training data [15,16]. This augmentation approach is not much beneficial for ASR system.…”

Data Augmentation Using Spectral Warping for Low Resource Children ASR

“…Such augmentations are only successful either through real time resource collection or by production of data which is similar to natural speech [8]. Other common approaches for data augmentation are prosody modification and speed/volume perturbation [2,[9][10][11]. Apart from these, in [12] spectral augmentation was used directly on Mel frequency cepstral coefficient (MFCC) features and filter bank features to improve ASR performance.…”