A Lightly Supervised Approach to Detect Stuttering in Children's Speech

Abstract

In speech pathology, new assistive technologies using ASR and machine learning approaches are being developed for detecting speech disorder events. Classically-trained ASR model tends to remove disfluencies from spoken utterances, due to its focus on producing clean and readable text output. However, diagnostic systems need to be able to track speech disfluencies, such as stuttering events, in order to determine the severity level of stuttering. To achieve this, ASR systems must be adapted to recognise full verbatim utterances, including pseudo-words and non-meaningful part-words. This work proposes a training regime to address this problem, and preserve a full verbatim output of stuttering speech. We use a lightly-supervised approach using task-oriented lattices to recognise the stuttering speech of children performing a standard reading task. This approach improved the WER by 27.8% relative to a baseline that uses word-lattices generated from the original prompt. The improved results preserved 63% of stuttering events (including sound, word, part-word and phrase repetition, and revision). This work also proposes a separate correction layer on top of the ASR that detects prolongation events (which are poorly recognised by the ASR). This increases the percentage of preserved stuttering events to 70%.

Detecting Multiple Speech Disfluencies Using a Deep Residual Network with Bidirectional Long Short-Term Memory

“…As a result, while these methods struggle with sub-word stutters such as sound repetition or revision, they perform well for word repetition or prolongation. This can be observed in Table 3 as [11] performs better than our method by a small margin (3.2%) for word repetition. Additionally, [11] performs with a lower miss rate than ours for detection of prolongation (5.92%).…”

“…Another model using Bi-LSTMs with condition random fields (CRFs) to get an average F-score of 85.9% across all stutter types [15]. The current state-of-the-art stutter classification method uses task-oriented finite state transducer (FST) lattices to detect repetition stutters with an average 37% miss rate across 4 different types of [11].…”

“…The results of our experiments for the UCLASS dataset is summarized in Table 3, where we compare our method to [11]. Additionally, to evaluate the need for bidirectional LSTM as opposed to a unidirectional LSTM, we compare our results to a baseline model where a ResNet with LSTM is used instead of our proposed model.…”

“…This is partially due to the fact that the notion of detecting and classifying the type and location of stutters can be a difficult problem, especially when factoring in variables such as gender, speech rate, accent, and phone-realization [8]. Existing works in the area mostly rely on automatic speech recognition (ASR) to first convert audio signals to text, and then utilize language models to detect and identify the stutters [9] [10] [11]. While this approach has proven effective and achieved promising results, the reliance on ASR can both be a potential source for error, as well as an unnecessary additional computational step.…”

SEP-28k: A Dataset for Stuttering Event Detection from Podcasts with People Who Stutter

“…The focus of this paper is on detection of five stuttering event types: Blocks, Prolongations, Sound Repetitions, Word/Phrase Repetitions, and Interjections. Existing work has explored this problem using traditional signal processing techniques [15,16,17], language modeling (LM) [12,18,19,20,21], and acoustic modeling (AM) [21,10]. Each approach has be shown to be effective 1.…”