Automatic Intelligibility Assessment of Dysarthric Speech Using Phonologically-Structured Sparse Linear Model

Kim, Myung Jong; Kim, Younggwan; Kim, Hoirin

doi:10.1109/taslp.2015.2403619

Cited by 40 publications

(25 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 5 If we consider the relationship between phonemes using weighted finite state transducer [54], which imposes different weights to phoneme mapping, we can obtain more reasonable phoneme alignment results even when we have a lot of phoneme errors. …”

mentioning

confidence: 99%

Speaker-Independent Silent Speech Recognition From Flesh-Point Articulatory Movements Using an LSTM Neural Network

Kim

Cao

Mau

et al. 2017

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Silent speech recognition (SSR) converts non-audio information such as articulatory movements into text. SSR has the potential to enable persons with laryngectomy to communicate through natural spoken expression. Current SSR systems have largely relied on speaker-dependent recognition models. The high degree of variability in articulatory patterns across different speakers has been a barrier for developing effective speaker-independent SSR approaches. Speaker-independent SSR approaches, however, are critical for reducing the amount of training data required from each speaker. In this paper, we investigate speaker-independent SSR from the movements of flesh points on tongue and lip with articulatory normalization methods that reduce the inter-speaker variation. To minimize the across-speaker physiological differences of the articulators, we propose Procrustes matching-based articulatory normalization by removing locational, rotational, and scaling differences. To further normalize the articulatory data, we apply feature-space maximum likelihood linear regression and i-vector. In this paper, we adopt a bidirectional long short term memory recurrent neural network (BLSTM) as an articulatory model to effectively model the articulatory movements with long-range articulatory history. A silent speech data set with flesh points was collected using an electromagnetic articulograph (EMA) from twelve healthy and two laryngectomized English speakers. Experimental results showed the effectiveness of our speaker-independent SSR approaches on healthy as well as laryngectomy speakers. In addition, BLSTM outperformed standard deep neural network. The best performance was obtained by BLSTM with all the three normalization approaches combined.

show abstract

mentioning

confidence: 99%

Speaker-Independent Silent Speech Recognition From Flesh-Point Articulatory Movements Using an LSTM Neural Network

Kim

Cao

Mau

et al. 2017

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…However, speaker recognition for dysarthric individuals is a challenging issue [3,4]. It is associated with unusual phonation and amplitude, especially on explosive phonemes [1].…”

Section: Introductionmentioning

confidence: 99%

“…It is associated with unusual phonation and amplitude, especially on explosive phonemes [1]. However, speaker recognition for dysarthric individuals is a challenging issue [3,4]. The human auditory system may not be able to successfully identify these people by their voices because of these unnatural variations.…”

Section: Introductionmentioning

confidence: 99%

Dysarthric speaker identification with different degrees of dysarthria severity using deep belief networks

et al. 2018

View full text Add to dashboard Cite

Dysarthria is a degenerative disorder of the central nervous system that affects the control of articulation and pitch; therefore, it affects the uniqueness of sound produced by the speaker. Hence, dysarthric speaker recognition is a challenging task. In this paper, a feature‐extraction method based on deep belief networks is presented for the task of identifying a speaker suffering from dysarthria. The effectiveness of the proposed method is demonstrated and compared with well‐known Mel‐frequency cepstral coefficient features. For classification purposes, the use of a multi‐layer perceptron neural network is proposed with two structures. Our evaluations using the universal access speech database produced promising results and outperformed other baseline methods. In addition, speaker identification under both text‐dependent and text‐independent conditions are explored. The highest accuracy achieved using the proposed system is 97.3%.

show abstract

“…A speaker-specific pronunciation dictionary was automatically generated using phoneme posterior probabilities of a deep neural network (DNN) trained on regular speech [21] or the state-specific vector of phone-cluster adaptive training-based acoustic models [49]. Weighted finite state transducers (WFSTs) using phonetic confusion matrices resulting from a regular ASR system were built to allow phonetic confusions during decoding process [22], [23], [32]. …”

Section: Introductionmentioning

confidence: 99%

Regularized Speaker Adaptation of KL-HMM for Dysarthric Speech Recognition

Kim

Yoo

et al. 2017

IEEE Trans. Neural Syst. Rehabil. Eng.

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of recognizing the speech uttered by patients with dysarthria, which is a motor speech disorder impeding the physical production of speech. Patients with dysarthria have articulatory limitation, and therefore, they often have trouble in pronouncing certain sounds, resulting in undesirable phonetic variation. Modern automatic speech recognition systems designed for regular speakers are ineffective for dysarthric sufferers due to the phonetic variation. To capture the phonetic variation, Kullback-Leibler divergence based hidden Markov model (KL-HMM) is adopted, where the emission probability of state is parametrized by a categorical distribution using phoneme posterior probabilities obtained from a deep neural network-based acoustic model. To further reflect speaker-specific phonetic variation patterns, a speaker adaptation method based on a combination of L2 regularization and confusion-reducing regularization which can enhance discriminability between categorical distributions of KL-HMM states while preserving speaker-specific information is proposed. Evaluation of the proposed speaker adaptation method on a database of several hundred words for 30 speakers consisting of 12 mildly dysarthric, 8 moderately dysarthric, and 10 non-dysarthric control speakers showed that the proposed approach significantly outperformed the conventional deep neural network based speaker adapted system on dysarthric as well as non-dysarthric speech.

show abstract

Automatic Intelligibility Assessment of Dysarthric Speech Using Phonologically-Structured Sparse Linear Model

Cited by 40 publications

References 32 publications

Speaker-Independent Silent Speech Recognition From Flesh-Point Articulatory Movements Using an LSTM Neural Network

Speaker-Independent Silent Speech Recognition From Flesh-Point Articulatory Movements Using an LSTM Neural Network

Dysarthric speaker identification with different degrees of dysarthria severity using deep belief networks

Regularized Speaker Adaptation of KL-HMM for Dysarthric Speech Recognition

Contact Info

Product

Resources

About