Regularized Speaker Adaptation of KL-HMM for Dysarthric Speech Recognition

Abstract

This paper addresses the problem of recognizing the speech uttered by patients with dysarthria, which is a motor speech disorder impeding the physical production of speech. Patients with dysarthria have articulatory limitation, and therefore, they often have trouble in pronouncing certain sounds, resulting in undesirable phonetic variation. Modern automatic speech recognition systems designed for regular speakers are ineffective for dysarthric sufferers due to the phonetic variation. To capture the phonetic variation, Kullback-Leibler divergence based hidden Markov model (KL-HMM) is adopted, where the emission probability of state is parametrized by a categorical distribution using phoneme posterior probabilities obtained from a deep neural network-based acoustic model. To further reflect speaker-specific phonetic variation patterns, a speaker adaptation method based on a combination of L2 regularization and confusion-reducing regularization which can enhance discriminability between categorical distributions of KL-HMM states while preserving speaker-specific information is proposed. Evaluation of the proposed speaker adaptation method on a database of several hundred words for 30 speakers consisting of 12 mildly dysarthric, 8 moderately dysarthric, and 10 non-dysarthric control speakers showed that the proposed approach significantly outperformed the conventional deep neural network based speaker adapted system on dysarthric as well as non-dysarthric speech.

Dysarthric Speech Recognition Using Convolutional LSTM Neural Network

“…Our experimental results demonstrated that CLSTM-RNN has the potential to improve the ASR performance as a speaker-independent acoustic model for the patients with ALS. To further improve the ASR accuracies, techniques for session/speaker variability compensation including acoustic feature transformation [25,26], acoustic model adaptation [27], and pronunciation variation modeling [27,28] can be further applied. We speculate that the results may improve once a larger training dataset from more ALS patients is obtained.…”

“…Our approach presents a possibility in effectively modeling dysarthric speech (even low intelligible speech) in a speaker-independent way. Future directions include 1) a test of the CLSTM-RNN approach using a larger dataset collected from more subjects, 2) applying speaker adaptation/normalization techniques [27], and 3) using articulatory information [25,29].…”

Adaptation Algorithms for Neural Network-Based Speech Recognition: An Overview

“…System References Model Hybrid [74], [75], [113], [141], [153], [154], [168], [178]- [180], [195], [212], [213], [230], [240], [284]-[290] E2E…”

Dysarthric Speech Recognition using Convolutional Recurrent Neural Networks

“…Korean Phonetically Optimized Words (KPOW) databas, Korean Phonetically Balanced Words (KPBW) database, Korean Phonetically Rich Words (KPRW) database and SI dysarthria adaptation were used for dysarthtic speech recognition with KL-HMM and compared with GMM-HMM and DNN-HMM. The framework of KL-HMM showed that is effective for dysarthric speakers to improve the performance [29].…”