Machine learning for stuttering identification: Review, challenges and future directions

Sheikh, Shakeel A.; Sahidullah, Md; Hirsch, Fabrice; Ouni, Slim

doi:10.1016/j.neucom.2022.10.015

Cited by 26 publications

(23 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the earlier work employed only a small set of disfluent speakers in their experimental studies, and has approached the SD problem as a binary classification problem: disfluent vs fluent identification or one vs other type [31]. The StutterNet we propose in our earlier work, is a time delay neural network based architecture that has been used to tackle the SD as a multi-class classification problem.…”

Section: A Stutternetmentioning

confidence: 99%

“…To evaluate the model performance, we use the following metrics: macro F1-score and accuracy which are the standard and are widely used in the stuttered speech domain [27], [28], [31], [36], [67]. The macro F1-score (F 1 ) (which combines the advantages of both precision and recall in a single metric unlike unweighted average recall which only takes recall into account) from equation ( 3) is often used in class imbalance scenarios with the intention to give equal importance to frequent and infrequent classes, and also is more robust towards the error type distribution [68].…”

Section: Evaluation Metricsmentioning

confidence: 99%

“…The disfluencies considered in this work are prolongations, repetitions, and interjections. A detailed summary and comparison of various feature extraction methods and classifiers can be found in [31].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning

Sheikh

Sahidullah

Hirsch³

et al. 2023

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Stuttering is a neuro-developmental speech impairment characterized by uncontrolled utterances (interjections) and core behaviors (blocks, repetitions, and prolongations), and is caused by the failure of speech sensorimotors. Due to its complex nature, stuttering detection (SD) is a difficult task. If detected at an early stage, it could facilitate speech therapists to observe and rectify the speech patterns of persons who stutter (PWS). The stuttered speech of PWS is usually available in limited amounts and is highly imbalanced. To this end, we address the class imbalance problem in the SD domain via a multibranching (MB) scheme and by weighting the contribution of classes in the overall loss function, resulting in a huge improvement in stuttering classes on the SEP-28k dataset over the baseline (StutterNet). To tackle data scarcity, we investigate the effectiveness of data augmentation on top of a multi-branched training scheme. The augmented training outperforms the MB StutterNet (clean) by a relative margin of 4.18% in macro F1-score (F 1 ). In addition, we propose a multi-contextual (MC) StutterNet, which exploits different contexts of the stuttered speech, resulting in an overall improvement of 4.48% in F 1 over the single context based MB StutterNet. Finally, we have shown that applying data augmentation in the cross-corpora scenario can improve the overall SD performance by a relative margin of 13.23% in F 1 over the clean training.

show abstract

Section: A Stutternetmentioning

confidence: 99%

Section: Evaluation Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning

Sheikh

Sahidullah

Hirsch³

et al. 2023

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…Studies show that persons who stutter (PWS) encounter several hardships in social and professional interactions (Kehoe and Contributors 2006). In addition, more people are progressively interacting with voice assistants, but they ignore and fail to recognize stuttered speech (Sheikh et al 2021a), and the stuttering detection (SD) can be exploited to improve automatic speech recognition (ASR) for PWS to access voice assistants such as Alexa, Siri, etc.…”

Section: Introductionmentioning

confidence: 99%

“…Usually, SD is addressed by various listening and brain scan tests (Ingham et al 1996;Smith and Weber 2017;Sheikh et al 2021a). However, this method of SD is high-priced and requires a demanding effort from speech therapists.…”

Section: Introductionmentioning

confidence: 99%

Stuttering detection using speaker representations and self-supervised contextual embeddings

Sheikh

Sahidullah²,

Hirsch

et al. 2023

Int J Speech Technol

View full text Add to dashboard Cite

The adoption of advanced deep learning architectures in stuttering detection (SD) tasks is challenging due to the limited size of the available datasets. To this end, this work introduces the application of speech embeddings extracted from pre-trained deep learning models trained on large audio datasets for different tasks. In particular, we explore audio representations obtained using emphasized channel attention, propagation, and aggregation time delay neural network (ECAPA-TDNN) and Wav2Vec2.0 models trained on VoxCeleb and LibriSpeech datasets respectively. After extracting the embeddings, we benchmark with several traditional classifiers, such as the K-nearest neighbour (KNN), Gaussian naive Bayes, and neural network, for the SD tasks. In comparison to the standard SD systems trained only on the limited SEP-28k dataset, we obtain a relative improvement of 12.08%, 28.71%, 37.9% in terms of unweighted average recall (UAR) over the baselines. Finally, we have shown that combining two embeddings and concatenating multiple layers of Wav2Vec2.0 can further improve the UAR by up to 2.60% and 6.32% respectively.

show abstract

Multi-label Dysfluency Classification

Jouaiti

Dautenhahn

2022

Speech and Computer

View full text Add to dashboard Cite

Machine learning for stuttering identification: Review, challenges and future directions

Cited by 26 publications

References 80 publications

Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning

Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning

Stuttering detection using speaker representations and self-supervised contextual embeddings

Multi-label Dysfluency Classification

Contact Info

Product

Resources

About