Automatic Hypernasality Detection in Cleft Palate Speech Using CNN

Wang, Xiyue; Tang, Ming; Yang, Sen; Yin, Han; Huang, Hua; He, Ling

doi:10.1007/s00034-019-01141-x

Cited by 15 publications

(31 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…support vector machines (SVMs), Gaussian mixture models (GMMS)) that detect hypernasal speech [15], [16], [18], [19]. Recently, convolutional neural network and recurrent neural networks have also been used for the same purpose [20], [21].…”

Section: A Related Workmentioning

confidence: 99%

“…Most of these automatic algorithms for detection of hypernasality were developed in a binary classification setting, i.e., healthy vs. hypernasal speech [15], [16], [18]- [21]. This is inconsistent with clinical practice, where clinicians require more fine-grained information (e.g.…”

Section: A Related Workmentioning

confidence: 99%

“…Hypernasality estimation based on supervised machine learning requires large, labeled speech corpora. Most of the existing speech-based hypernasality evaluation methods use speech samples and corresponding perceptual ratings to train machine learning models, including k-near neighborhood classifier [25], Gaussian mixture models [24], support vector machines [18], [19], [23], [30], and deep neural networks [20], [21], [31]. The performance of these systems critically depends on the availability of clinical hypernasal speech databases that include speech samples from patients and corresponding hypernasality ratings from trained SLPs.…”

Section: A Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Deep Learning Algorithm for Objective Assessment of Hypernasality in Children with Cleft Palate

Mathad¹,

Scherer²,

Chapman³

et al. 2020

Preprint

View full text Add to dashboard Cite

Evaluation of hypernasality requires extensive perceptual training by clinicians and extending this training on a large scale internationally is untenable; this compounds the health disparities that already exist among children with cleft. In this work, we present the objective hypernasality measure (OHM), a speech analytics algorithm that automatically measures hypernasality in speech, and validate it relative to a group of trained clinicians. Methods: We trained a deep neural network (DNN) on approximately 100 hours of a publiclyavailable healthy speech corpus to detect the presence of nasal acoustic cues generated through the production of nasal consonants and nasalized phonemes in speech. Importantly, this model does not require any clinical data for training. The posterior probabilities of the deep learning model were aggregated at the sentence and speaker-levels to compute the OHM. Results: The results showed that the OHM was significantly correlated with the perceptual hypernasality ratings in the Americleft database (r=0.797, p<0.001), and with the New Mexico Cleft Palate Center (NMCPC) database (r=0.713, p<0.001). In addition, we evaluated the relationship between the OHM and articulation errors; the sensitivity of the OHM in detecting the presence of very mild hypernasality; and establishing the internal reliability of the metric. Further, the performance of OHM was compared with a DNN regression algorithm directly trained on the hypernasal speech samples. Significance: The results indicate that the OHM is able to rate the severity of hypernasality on par with Americlefttrained clinicians on this dataset.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Learning Algorithm for Objective Assessment of Hypernasality in Children with Cleft Palate

Mathad¹,

Scherer²,

Chapman³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The original gradient descent optimization algorithm converges slowly near the bottom of a slope but this is avoided when the AdaGrad algorithm is used. Because of the steep directional gradient, the learning rate will decay faster, which is conducive to the parameter moving in the direction closer to the bottom of the slope, thus accelerating the convergence [46].…”

Section: Proposed Lstm-dnn Algorithmmentioning

confidence: 99%

Walking Gait Phase Detection Based on Acceleration Signals Using LSTM-DNN Algorithm

2019

View full text Add to dashboard Cite

Gait phase detection is a new biometric method which is of great significance in gait correction, disease diagnosis, and exoskeleton assisted robots. Especially for the development of bone assisted robots, gait phase recognition is an indispensable key technology. In this study, the main characteristics of the gait phases were determined to identify each gait phase. A long short-term memory-deep neural network (LSTM-DNN) algorithm is proposed for gate detection. Compared with the traditional threshold algorithm and the LSTM, the proposed algorithm has higher detection accuracy for different walking speeds and different test subjects. During the identification process, the acceleration signals obtained from the acceleration sensors were normalized to ensure that the different features had the same scale. Principal components analysis (PCA) was used to reduce the data dimensionality and the processed data were used to create the input feature vector of the LSTM-DNN algorithm. Finally, the data set was classified using the Softmax classifier in the full connection layer. Different algorithms were applied to the gait phase detection of multiple male and female subjects. The experimental results showed that the gait-phase recognition accuracy and F-score of the LSTM-DNN algorithm are over 91.8% and 92%, respectively, which is better than the other three algorithms and also verifies the effectiveness of the LSTM-DNN algorithm in practice.

show abstract

“…More recently, hypernasality evaluation algorithms rely on machine learning; frame-level features such as MFCCs are extracted from the segmented regions to train classifiers like support vector machine and Gaussian mixture models [7,8,9]. In similar vein, convolutional neural networks and recurrent neural networks have also been used to detect hypernasality [10,11].…”

Section: Introductionmentioning

confidence: 99%

An Attention Model for Hypernasality Prediction in Children with Cleft Palate

Mathad

Scherer

Chapman

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Hypernasality refers to the perception of abnormal nasal resonances in vowels and voiced consonants. Estimation of hypernasality severity from connected speech samples involves learning a mapping between the frame-level features and utterance-level clinical ratings of hypernasality. However, not all speech frames contribute equally to the perception of hypernasality. In this work, we propose an attention-based bidirectional long-short memory (BLSTM) model that directly maps the frame-level features to utterance-level ratings by focusing only on specific speech frames carrying hypernasal cues. The models performance is evaluated on the Americleft database containing speech samples of children with cleft palate and clinical ratings of hypernasality. We analyzed the attention weights over broad phonetic categories and found that the model yields results consistent with what is known in the speech science literature. Further, the correlation between the predicted and perceptual rating is found to be significant (r = 0.684, p < 0.001) and better than conventional BLSTMs trained using frame-wise and last-frame approaches.

show abstract

Automatic Hypernasality Detection in Cleft Palate Speech Using CNN

Cited by 15 publications

References 61 publications

A Deep Learning Algorithm for Objective Assessment of Hypernasality in Children with Cleft Palate

A Deep Learning Algorithm for Objective Assessment of Hypernasality in Children with Cleft Palate

Walking Gait Phase Detection Based on Acceleration Signals Using LSTM-DNN Algorithm

An Attention Model for Hypernasality Prediction in Children with Cleft Palate

Contact Info

Product

Resources

About