Development of the CUHK Dysarthric Speech Recognition System for the UA Speech Corpus

Yu, Jianwei; Xie, Xurong; Liu, Shansong; Hu, Shoukang; Lam, Man Fai; Wu, Xixin; Wong, Ka Ho; Liu, Xunying; Meng, Helen

doi:10.21437/interspeech.2018-1541

Cited by 47 publications

(54 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Block 1 (B1) and block 3 (B3) were treated as the training set, leaving the block 2 (B2) as the test set. The feed forward DNN based acoustic model without using pitch features is determined as the baseline system, since more advanced forms of neural networks based acoustic models such as time delayed neural networks [39] and long short-term memory recurrent neural networks [40] did not produce lower WER over feed forward DNN on the UASpeech recognition task [23]. All the investigated neural network models on the UASpeech corpus were built in Pytorch [41].…”

Section: Incorporating Pitch Featurementioning

confidence: 99%

“…Speech recognition for disordered speech is a challenging task in general [19]. Acoustic features play a major role in early and recent studies of disordered speech recognition [20,21,22,23]. To the best of our knowledge, the vast majority of speech recognition systems using pitch features are conducted on normal speech, very limited research incorporating pitch features has been found for disordered speech.…”

Section: Introductionmentioning

confidence: 99%

“…The average WER of the DNN ASR systems[23] built by HTK on the 16 speakers is 33.9%. The authors of[23] provide the speaker level WERs. Our baseline DNN ASR systems produced a competitive result.…”

mentioning

confidence: 99%

See 2 more Smart Citations

On the Use of Pitch Features for Disordered Speech Recognition

Liu

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Pitch features have long been known to be useful for recognition of normal speech. However, for disordered speech, the significant degradation of voice quality renders the prosodic features, such as pitch, not always useful, particularly when the underlying conditions, for example, damages to the cerebellum, introduce a large effect on prosody control. Hence, both acoustic and prosodic information can be distorted. To the best of our knowledge, there has been very limited research on using pitch features for disordered speech recognition. In this paper, a comparative study of multiple approaches designed to incorporate pitch features is conducted to improve the performance of two disordered speech recognition tasks: English UASpeech, and Cantonese CUDYS. A novel gated neural network (GNN) based approach is used to improve acoustic and pitch feature integration over a conventional concatenation between the two. Bayesian estimation of GNNs is also investigated to further improve their robustness.

show abstract

Section: Incorporating Pitch Featurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the Use of Pitch Features for Disordered Speech Recognition

Liu

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition to traditional Gaussian mixture model (GMM)hidden Markov model (HMM) systems [6], deep neural networks have been applied to robust acoustic modeling of dysarthric speech recognition systems in recent years. For example, Kim et al [7] captured the distinct characteristics of dysarthric speech by taking advantage of convolutional neural network (CNN) for extracting effective local features and recurrent neural network (RNN) for modeling temporal dependencies of the acoustic features.…”

Section: Introductionmentioning

confidence: 99%

Multi-Task Transformer with Input Feature Reconstruction for Dysarthric Speech Recognition

Ding

Sun

Zhao

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Dysarthria is a motor speech disorder caused by damage to the part of the nervous system that controls the physical production of speech. It poses great challenges in building robust dysarthric speech recognition (DSR) due to the high inter-and intra-speaker variability. To this end, we propose a multi-task Transformer with input feature reconstruction as an auxiliary task, where the main task of DSR and the auxiliary reconstruction task share the same encoder network. The auxiliary task aims to reconstruct clear speech features from corrupted speech of healthy speakers (intra-domain) or dysarthric speakers (cross-domain). Further, to alleviate the imbalanced distribution of dysarthria data sets, we devise an adaptive rebalance sampling scheme to improve the utterance sampling frequency of dysarthric speech. Experimental results show that the proposed model considerably outperforms other baselines across speakers with varying severity of dysarthria.

show abstract

“…For instance, to reduce the influence caused by dysarthric speech variability in tandem ASR systems, bottleneck features have been proposed via DNNs using a large amount out-of-domain data [13], or convolutional neural networks have been employed [14]. Various advanced forms of DNN architecture were tested in [15] for both tandem and hybrid ASR systems for dysarthric speech. To fully exploit DNNbased acoustic modelling, data augmentation was successfully applied based on speed and tempo perturbation in the signal domain for dysarthric speech recognition [16].…”

Section: Introductionmentioning

confidence: 99%

Source Domain Data Selection for Improved Transfer Learning Targeting Dysarthric Speech Recognition

Xiong

Barker

Yue

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper presents an improved transfer learning framework applied to robust personalised speech recognition models for speakers with dysarthria. As the baseline of transfer learning, a state-of-theart CNN-TDNN-F ASR acoustic model trained solely on source domain data is adapted onto the target domain via neural network weight adaptation with the limited available data from target dysarthric speakers. Results show that linear weights in neural layers play the most important role for an improved modelling of dysarthric speech evaluated using UASpeech corpus, achieving averaged 11.6% and 7.6% relative recognition improvement in comparison to the conventional speaker-dependent training and data combination, respectively. To further improve the transferability towards target domain, we propose an utterance-based data selection of the source domain data based on the entropy of posterior probability, which is analysed to statistically obey a Gaussian distribution. Compared to a speaker-based data selection via dysarthria similarity measure, this allows for a more accurate selection of the potentially beneficial source domain data for either increasing the target domain training pool or constructing an intermediate domain for incremental transfer learning, resulting in a further absolute recognition performance improvement of nearly 2% added to transfer learning baseline for speakers with moderate to severe dysarthria.

show abstract

Development of the CUHK Dysarthric Speech Recognition System for the UA Speech Corpus

Cited by 47 publications

References 0 publications

On the Use of Pitch Features for Disordered Speech Recognition

On the Use of Pitch Features for Disordered Speech Recognition

Multi-Task Transformer with Input Feature Reconstruction for Dysarthric Speech Recognition

Source Domain Data Selection for Improved Transfer Learning Targeting Dysarthric Speech Recognition

Contact Info

Product

Resources

About