Articulatory Knowledge in the Recognition of Dysarthric Speech

Abstract

Abstract-Disabled speech is not compatible with modern generative and acoustic-only models of speech recognition (ASR). This work considers the use of theoretical and empirical knowledge of the vocal tract for atypical speech in labeling segmented and unsegmented sequences. These combined models are compared against discriminative models such as neural networks, support vector machines, and conditional random fields. Results show significant improvements in accuracy over the baseline through the use of production knowledge. Furthermore, although the statistics of vocal tract movement do not appear to be transferable between regular and disabled speakers, transforming the space of the former given knowledge of the latter before retraining gives high accuracy. This work may be applied within components of assistive software for speakers with dysarthria.

Recognizing Dysarthric Speech due to Amyotrophic Lateral Sclerosis with Across-Speaker Articulatory Normalization

“…The histogram of ALS patients follows general trend of warping factor distribution for females (typically < 1.0) and males (typically > 1.0). Figures 5,6,7,and 8 give the PERs of speaker-independent dysarthric (due to ALS) speech recognition results using different context models and recognizers, respectively: (1) monophone GMM-HMM, (2) triphone GMM-HMM, (3) monophone DNN-HMM, and (4) triphone DNN-HMM with individual or combinations of VTLN, Procrustes matching, and fMLLR. These results suggest that VTLN, Procrustes matching, and fMLLR were all effective for speaker-independent dysarthric speech recognition from acoustic data, articulatory data, or combined.…”

“…Based on the recent literature on speech recognition with articulatory data (e.g., [7,[14][15][16][17][18][19][20]), we hypothesized the followings for dysarthric speech recognition for ALS: 1) adding articulatory data (collected from ALS patients) would improve the speech recognition performance, 2) feature normalization in articulatory, acoustic, and both spaces is critical and necessary for speaker-independent dysarthric speech recognition with articulatory data, and 3) recent state-of-the-art approach, deep neural network (DNN)-hidden Markov model (HMM) would outperform the long-standing approach, Gaussian mixture model (GMM)-HMM.…”

“…To reduce speaker-specific difference, researchers have tried different approaches to normalize the articulatory movements including data-driven approaches (e.g., principal component analysis [7]) or physiological approaches including aligning the tongue position when producing vowels [24][25][26], consonants [27,28], and pseudo-words [29] to a reference (e.g., palate [24,25], or a general tongue shape [27] Procrustes matching, a bidimensional shape analysis technique [30], has been used to minimize the translational, scaling, and rotational effects of articulatory data across speakers [28,29,31]. Recent studies indicated Procrustes matching was effective for speaker-independent silent speech recognition (i.e., recognizing speech from articulatory data only) [18,19].…”

A cross-lingual adaptation approach for rapid development of speech recognizers for learning disabled users

“…For example, [17] studied the predictability of articulatory errors and trained a Bayesian network in order to build an augmented ASR system that considered the statistical relationships between vocal tract configurations and their acoustic consequences. Similarly, [18] focused on aspects of syllabic strength for moderate hypokinetic dysarthric speech.…”

Isolated Word Recognition for Dysarthric Patients

“…HMMs are a broad class of doubly stochastic models for non-stationary signals that can be inserted in to other stochastic models to incorporate information from several hierarchical knowledge sources [6].An HMM-based recognizer using variable duration of Hamming window is proposed in [7] that raises the recognition rate of dysarthric speech up to 80 %. Discrete articulatory feature recognition has been applied to identify values for concurrent features in [8]. Here, articulatory features are collected into different categories, each with a number of possible values.…”