Automatic Annotation of Disfluent Speech in Children’s Reading Tasks

Proença, Jorge; Celorico, Dirce; Lopes, Carla; Candeias, Sara; Perdigão, Fernando

doi:10.1007/978-3-319-49169-1_17

Cited by 3 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The following subsections will each describe the applications of the three different miscue types used in our study. These miscues were introduced in Proença et al [9,19,20]. EXT is a miscue in which one phoneme is abnormally extended when reading a text.…”

Section: Modified-srs Corpus Using Data Augmentationmentioning

confidence: 99%

Automatic Miscue Detection Using RNN Based Models with Data Augmentation

Hong¹,

Gweon

2018

Interspeech 2018

View full text Add to dashboard Cite

This study proposes a method of using data augmentation to address the problem of data shortages in miscue detection tasks. Three main steps were taken. First, a phoneme classifier was developed to acquire force-aligned data, which would be used for miscue classification and data augmentation. In order to create the phoneme classifier, phonetic features of "Seoul Reading Speech" (SRS) corpus were extracted by using grapheme-to-phoneme (G2P) to train CNN-based models. Second, to obtain miscue labeled corpus, we performed data augmentation using the phoneme classifier output, which is artificially generated miscue corpus of SRS (modified-SRS). This miscue corpus was created by randomly deleting or modifying sound sections according to three miscue categories; extension (EXT), pause (PAU), and pre-correction (PRE). Third, the performance of the miscue classifier was tested after training three types of RNN based models (LSTM, BiLSTM, BiGRU) with the modified-SRS corpus. The results show that the BiGRU model performed best at 0.819 in F1-score on augmented data, while BiLSTM model performed best at 0.512 on real data.

show abstract

Section: Modified-srs Corpus Using Data Augmentationmentioning

confidence: 99%

Automatic Miscue Detection Using RNN Based Models with Data Augmentation

Hong¹,

Gweon

2018

Interspeech 2018

View full text Add to dashboard Cite

show abstract

“…The employed methods to automatically annotate an utterance while detecting reading disfluencies follow two stages, and are described in further detail in [17]. The first is segmentation, providing candidate segments for words.…”

Section: Automatic Annotationmentioning

confidence: 99%

“…The same features are calculated for the two reading tasks separately (sentences and pseudowords), doubling the number of features. Features can be split into four groups: reading speed (1-6), silence related features (7-9), rate of disfluencies (10)(11)(12)(13)(14) and task meta-information (15)(16)(17)(18), where difficulty is based on a set of rules for doubtful or harder pronunciations [16]. Since deletions and insertions are not detected by the automatic annotation, FastR is not computed for that case.…”

Section: Automatic Annotationmentioning

confidence: 99%

Automatic Evaluation of Children Reading Aloud on Sentences and Pseudowords

et al. 2017

Self Cite

View full text Add to dashboard Cite

Reading aloud performance in children is typically assessed by teachers on an individual basis, manually marking reading time and incorrectly read words. A computational tool that assists with recording reading tasks, automatically analyzing them and providing performance metrics could be a significant help. Towards that goal, this work presents an approach to automatically predicting the overall reading aloud ability of primary school children (6-10 years old), based on the reading of sentences and pseudowords. The opinions of primary school teachers were gathered as ground truth of performance, who provided 0-5 scores closely related to the expectations at the end of each grade. To predict these scores automatically, features based on reading speed and number of disfluencies were extracted, after an automatic disfluency detection. Various regression models were trained, with Gaussian process regression giving best results for automatic features. Feature selection from both sentence and pseudoword reading tasks gave the closest predictions, with a correlation of 0.944. Compared to the use of manual annotation with the best correlation being 0.952, automatic annotation was only 0.8% worse. Furthermore, the error rate of predicted scores relative to ground truth was found to be smaller than the deviation of evaluators' opinion per child.

show abstract

“…Performance metrics based on reading attempts include correct words per minute and rates of reading errors. Detecting all the different types of reading events has also been useful for the automatic annotation of speech databases [12].…”

Section: Introductionmentioning

confidence: 99%

“…Compared to our previous approaches [12], [21], we propose here a new decoding strategy using only syllables in a semiconstrained way. We also use and define multiple features for mispronunciation classification, which we believe is novel in this scope.…”

Section: Introductionmentioning

confidence: 99%

Mispronunciation Detection in Children's Reading of Sentences

Proença

Lopes

Tjalve

et al. 2018

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

This paper proposes an approach to automatically parse children's reading of sentences by detecting word pronunciations and extra content, and to classify words as correctly or incorrectly pronounced. This approach can be directly helpful for automatic assessment of reading level or for automatic reading tutors, where a correct reading must be identified. We propose a first segmentation stage to locate candidate word pronunciations based on allowing repetitions and false starts of a word's syllables. A decoding grammar based solely on syllables allows silence to appear during a word pronunciation. At a second stage, word candidates are classified as mispronounced or not. The feature that best classifies mispronunciations is found to be the log-likelihood ratio between a free phone loop and a word spotting model in the very close vicinity of the candidate segmentation. Additional features are combined in multifeature models to further improve classification, including: normalizations of the log-likelihood ratio, derivations from phone likelihoods, and Levenshtein distances between the correct pronunciation and recognized phonemes through two phoneme recognition approaches. Results show that most extra events were detected (close to 2% word error rate achieved) and that using automatic segmentation for mispronunciation classification approaches the performance of manual segmentation. Although the log-likelihood ratio from a spotting approach is already a good metric to classify word pronunciations, the combination of additional features provides a relative reduction of the miss rate of 18% (from 34.03% to 27.79% using manual segmentation and from 35.58% to 29.35% using automatic segmentation, at constant 5% false alarm rate).

show abstract

Automatic Annotation of Disfluent Speech in Children’s Reading Tasks

Cited by 3 publications

References 15 publications

Automatic Miscue Detection Using RNN Based Models with Data Augmentation

Automatic Miscue Detection Using RNN Based Models with Data Augmentation

Automatic Evaluation of Children Reading Aloud on Sentences and Pseudowords

Mispronunciation Detection in Children's Reading of Sentences

Contact Info

Product

Resources

About