Detection of Mispronunciations and Disfluencies in Children Reading Aloud

Abstract

To automatically evaluate the performance of children reading aloud or to follow a child's reading in reading tutor applications, different types of reading disfluencies and mispronunciations must be accounted for. In this work, we aim to detect most of these disfluencies in sentence and pseudoword reading. Detecting incorrectly pronounced words, and quantifying the quality of word pronunciations, is arguably the hardest task. We approach the challenge as a two-step process. First, a segmentation using task-specific lattices is performed, while detecting repetitions and false starts and providing candidate segments for words. Then, candidates are classified as mispronounced or not, using multiple features derived from likelihood ratios based on phone decoding and forced alignment, as well as additional meta-information about the word. Several classifiers were explored (linear fit, neural networks, support vector machines) and trained after a feature selection stage to avoid overfitting. Improved results are obtained using feature combination compared to using only the log likelihood ratio of the reference word (22% versus 27% miss rate at constant 5% false alarm rate).

A Lightly Supervised Approach to Detect Stuttering in Children's Speech

“…It has also been used to align and correct approximate transcriptions of long audio recordings [30], and for audio indexing and displaying subtitles. In a tutoring application, this approach was used by [31] to track and align a child's read passage.…”

Automatic Miscue Detection Using RNN Based Models with Data Augmentation

“…Using less reduced data contributes to performance improvement in speech recognition tasks [13]. Therefore, we expect similar performance improvement in classifying phonemes compared to existing models based on hidden Markov model or the maximum entropy model that use MFCC [9,12].…”

“…Therefore, we first created a phoneme classifier that can provide aligned phoneme labels given just a speech file. Unlike existing research that detects miscues with miscue annotated corpus [9], our approach starts the miscue detection task using speech files with only transcriptions, but no aligned phonemes. Figure 1 shows our phoneme classification pipeline as well as the data augmentation process.…”

“…Unlike existing works on binary miscue detection task [7,8], we need miscue types that occur in natural speech of reading. Since our goal is to classify miscue types for reading corpus at the phoneme level, we selected miscue types [9] that are applicable at the phoneme level. The three selected types are extension, pause, and precorrection.…”

Evaluation of reading performance of primary school children: Objective measurements vs. subjective ratings

“…In [16], the problems raised by the accurate and robust recognition of children's speech is reported. More recently, Proença et al [17] propose another specialized ASR system with the same goal. Both publications are more involved in solving the problem of the recognition of disfluent speech than in correlating results with subjective scores of the quality of the reading.…”