ASR-Based Evaluation and Feedback for Individualized Reading Practice

Bai, Yu; Hubers, Ferdy; Cucchiarini, Catia; Strik, Helmer

doi:10.21437/interspeech.2020-2842

Cited by 5 publications

(20 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We investigated which words the children found to be most difficult to read and how the explicit feedback provided by the reading tutor contributed to improving reading accuracy and speed in these Dutch first graders. Improvements in reading accuracy and speed were seen in different types of exercises [9]. After analyzing the data of this first experiment, we saw that a higher agreement between the system's scores and the judgements assigned by a human annotator could be obtained by adjusting the measures and thresholds in the system [10].…”

Section: Introductionmentioning

confidence: 87%

“…In a previous study we reported on research aimed at developing a reading tutor for Dutch first graders that employs ASR to "listen" and give feedback on pupils' reading aloud [9]. This system also has the capability of storing user activities and ASR information on the server.…”

Section: Methodsmentioning

confidence: 99%

“…The ASR training model used in the second experiment was similar to the one we used for the first experiment [9], the only differences being a) that now the mean of the probabilities of all the phones is taken as a measure instead of the lowest phone score and b) that the threshold was increased in order to the system to make judgements closer to human teachers.…”

Section: Asr Technologymentioning

confidence: 99%

“…Subsequently, they are instructed to read all words and sentences again (irrespective of their reading accuracy at the first attempt). For a more detailed description of the reading tutor, see [9].…”

Section: Reading Tutormentioning

confidence: 99%

See 3 more Smart Citations

Automatic Speech Recognition Technology and Reading Skill Development in Primary School

Bai¹,

Tejedor-Garcı́a²,

Hubers³

et al. 2021

ICERI2021 Proceedings

Self Cite

View full text Add to dashboard Cite

Reading is a learned skill that children acquire through instruction and practice. A desirable feature of that practice is that children can read aloud under the guidance of a teacher. Unfortunately, this is not always possible to a sufficient extent because of general time-constraints in teacher-fronted education. For this reason, experts have been looking at Automatic Speech Recognition (ASR) technology as a possible alternative for the "listening ear" that is usually lent by teachers. Especially for reading in English, the contribution of ASR technology has been investigated from various perspectives and this research even led to relatively successful commercial products. In general, in these products the aim for ASR technology is to follow the children while reading aloud and provide some form of support when they hesitate.An important requirement for this kind of research and applications is that ASR of child speech is of sufficient quality. In turn this requires that the ASR algorithms are trained with large amounts of child speech recordings that, in general, are difficult to obtain.A compounding problem with child speech recognition is that children, as a group, display an enormous amount of variation in terms of speech characteristics, which is in part related to their variable physical characteristics. This means that large amounts of speech recordings are needed for each age cohort, which makes achieving high-quality levels of ASR performance even more difficult.As a matter of fact, the reason why so far research and applications have mostly addressed English is that for English larger amounts of child speech recordings could be obtained than for other languages. As a way of circumventing the data sparsity problem, speech technologists have investigated child speech recognition by applying several techniques that had been developed for low-resourced languages.In general, research has shown that the younger the children, the more difficult it is to achieve good ASR performance. On the other hand, one can imagine that ASR-based applications for learning to read are especially required for young children.In this paper we report on research aimed at developing and testing a Dutch ASR-based reading tutor. Innovative features of this system are that it is intended for young children in the earlier stages of learning to read, when they start developing decoding skills, and that it addresses reading in Dutch. A preliminary study conducted during the pandemic when children were primarily working from home showed encouraging results. In the present paper we report on a recent, larger study in which children used the system in the classroom. After collecting over 617 thousand words from 263 first graders from 44 different schools, we analyzed various attempts by the children at reading words and sentences before and after they received various types of ASR-based feedback. The results indicate that successive attempts were characterized by improved reading accuracy and fluency. Significant improvement in accuracy ...

show abstract

Section: Introductionmentioning

confidence: 87%

Section: Methodsmentioning

confidence: 99%

Section: Asr Technologymentioning

confidence: 99%

Section: Reading Tutormentioning

confidence: 99%

See 2 more Smart Citations

Automatic Speech Recognition Technology and Reading Skill Development in Primary School

Bai¹,

Tejedor-Garcı́a²,

Hubers³

et al. 2021

ICERI2021 Proceedings

Self Cite

View full text Add to dashboard Cite

show abstract

“…For this purpose we employed various metrics, including Cohen's Kappa, Matthews Correlation Coefficient (MCC), precision, recall and F-measures. The material used in the reading tutor contains four reading tasks designed with different goals and difficulty levels for first graders [2]. We also investigated the ASR accuracy in each reading task and different word types from two reading tasks which contain both words that are familiar to the first graders and new words.…”

Section: Introductionmentioning

confidence: 99%

An ASR-Based Tutor for Learning to Read: How to Optimize Feedback to First Graders

Bai¹,

Tejedor-Garcı́a²,

Hubers³

et al. 2021

Speech and Computer

Self Cite

View full text Add to dashboard Cite

The interest in employing automatic speech recognition (ASR) in applications for reading practice has been growing in recent years. In a previous study, we presented an ASR-based Dutch reading tutor application that was developed to provide instantaneous feedback to first-graders learning to read. We saw that ASR has potential at this stage of the reading process, as the results suggested that pupils made progress in reading accuracy and fluency by using the software. In the current study, we used children's speech from an existing corpus (JAS-MIN) to develop two new ASR systems, and compared the results to those of the previous study. We analyze correct/incorrect classification of the ASR systems using human transcripts at word level, by means of evaluation measures such as Cohen's Kappa, Matthews Correlation Coefficient (MCC), precision, recall and F-measures. We observe improvements for the newly developed ASR systems regarding the agreement with human-based judgment and correct rejection (CR). The accuracy of the ASR systems varies for different reading tasks and word types. Our results suggest that, in the current configuration, it is difficult to classify isolated words. We discuss these results, possible ways to improve our systems and avenues for future research.

show abstract

An ASR-based Reading Tutor for Practicing Reading Skills in the First Grade: Improving Performance through Threshold Adjustment

Bai¹,

Hubers²,

Cucchiarini³

et al. 2021

IberSPEECH 2021

View full text Add to dashboard Cite

Automatic Speech Recognition (ASR) technology can potentially be employed to provide intensive practice and feedback to young children learning to read. So far there has been limited research on the use of ASR in the early stages of learning to read when children are still developing decoding skills. For this purpose, we developed an ASR-based system equipped with logging capabilities that can evaluate decoding skills in Dutch first graders reading aloud and provide them with instantaneous feedback. In a previous study we found that ASR-based feedback led to improved reading accuracy and speed, and that useful information could be obtained from the log-files. For the present paper we conducted thorough analyses of the data obtained with this ASR-based system by comparing it to human annotations of the same read aloud 11849 words from 38 pupils. We present the results of our analyses, and discuss how they can contribute to better and more personalized ASR-based reading instruction.

show abstract

ASR-Based Evaluation and Feedback for Individualized Reading Practice

Cited by 5 publications

References 13 publications

Automatic Speech Recognition Technology and Reading Skill Development in Primary School

Automatic Speech Recognition Technology and Reading Skill Development in Primary School

An ASR-Based Tutor for Learning to Read: How to Optimize Feedback to First Graders

An ASR-based Reading Tutor for Practicing Reading Skills in the First Grade: Improving Performance through Threshold Adjustment

Contact Info

Product

Resources

About