UT-Vocal Effort II: Analysis and constrained-lexicon recognition of whispered speech

Abstract

This study focuses on acoustic variations in speech introduced by whispering, and proposes several strategies to improve robustness of automatic speech recognition of whispered speech with neutral-trained acoustic models. In the analysis part, differences in neutral and whispered speech captured in the UT-Vocal Effort II corpus are studied in terms of energy, spectral slope, and formant center frequency and bandwidth distributions in silence, voiced, and unvoiced speech signal segments. In the part dedicated to speech recognition, several strategies involving front-end filter bank redistribution, cepstral dimensionality reduction, and lexicon expansion for alternative pronunciations are proposed. The proposed neutral-trained system employing redistributed filter bank and reduced features provides a 7.7 % absolute WER reduction over the baseline system trained on neutral speech, and a 1.3 % reduction over a baseline system with whisper-adapted acoustic models.

Deep neural network training for whispered speech recognition using small databases and generative model sampling

“…As shown in the third column of Table 4, the recognizer reaches a phoneme error rate (PER) of 27.5% on the neutral task and 45.0% on the whisper task. The results are in agreement with the results in Ghaffarzadegan et al (2014b), where whisper-trained recognizer tested on whisper data yielded a lower recognition rate compared to a neutral-trained model tested on neutral data. As discussed in Ghaffarzadegan et al (2014b), the main cause of this disparity may be attributed to the higher confusability of the whisper phone set where voiced and unvoiced phone groups seen in neutral speech are now all mapped to the unvoiced acoustic space, causing wide overlaps of originally voiced and unvoiced fricative and stop pairs.…”

Transfer Learning with Bottleneck Feature Networks for Whispered Speech Recognition

“…This degradation is not as pronounced if an AM trained from whispered speech were to be used to recognize neutral speech. This result has been consistent for experiments done in several languages for which there is a sizeable corpus of parallel whispered speech, such as Serbian [3], Japanese [4], Mandarin [5] and English [6,7]. The pattern is also consistent for different types of models, both Gaussian Mixture Models (GMM) trained with generative or discriminative methods, and even for Deep Neural Network (DNN) based AMs.…”

Whispered Speech Recognition using Hidden Markov Models and Support Vector Machines

“…The use of Teager energy cepstral coefficients with deep denoising autoencoder (DDA) has recently brought many benefits in speaker dependent (SD) neutraltrained whisper recognition [18]. Likewise, performances of speaker independent (SI) recognition of whispered speech have been significantly improved after adapting the acoustic model toward the DDA pseudo-whisper samples, compared to the model adaptation on an available small whisper set (for UT-Vocal Effort II speech corpus) [13,15]. However, to the best of our knowledge, comparison of different speech recognition tools that include SVM in recognition of whispered speech was not reported.…”