Detecting Mispronunciations of L2 Learners and Providing Corrective Feedback Using Knowledge-Guided and Data-Driven Decision Trees

Abstract

We propose a novel decision tree based framework to detect phonetic mispronunciations produced by L2 learners caused by using inaccurate speech attributes, such as manner and place of articulation. Compared with conventional score-based CAPT (computer assisted pronunciation training) systems, our proposed framework has three advantages: (1) each mispronunciation in a tree can be interpreted and communicated to the L2 learners by traversing the corresponding path from a leaf node to the root node; (2) corrective feedback based on speech attribute features, which are directly used to describe how consonants and vowels are produced using related articulators, can be provided to the L2 learners; and (3) by building the phone-dependent decision tree, the relative importance of the speech attribute features of a target phone can be automatically learned and used to distinguish itself from other phones. This information can provide L2 learners speech attribute feedback that is ranked in order of importance. In addition to the abovementioned advantages, experimental results confirm that the proposed approach can detect most pronunciation errors and provide accurate diagnostic feedback.

A Study of Objective Measurement of Comprehensibility through Native Speakers' Shadowing of Learners' Utterances

“…To provide those situations of oral interaction technically to learners, dialogue-based CALL (Computer Aided Language Learning) systems have been developed [1,2,3], where not only pronunciation errors but also grammatical errors can be detected and their corrective feedback is also provided. To assess learners' pronunciation, native speakers' acoustic models are often referred to and comparison is made between learners' speech and its corresponding native model.…”

Mispronunciation Diagnosis of L2 English at Articulatory Level Using Articulatory Goodness-Of-Pronunciation Features

“…As can be seen in Section 4, the number of the correctly pronounced phone is much larger than the number of mispronounced, which could make trained models biased. To prevent the bias problem, we adopt other phones' correctly pronounced observations as mispro-nounced samples of the target phone as much as the difference between the number of correct instances and the number of incorrect instances to make a balance [5].…”

“…There have been several studies to detect pronunciation errors of learners [2][3][4] [5]. The study of [2] suggested an extended recognition network (ERN), which expands pronunciation dictionaries of learners by predicting frequent erroneous pronunciation sequences.…”

“…The study of Ryu and Chung [10] proposes articulatory Goodness-Of-Pronunciations (aGOPs) as novel features for pronunciation assessment in English spoken by Korean learners. Furthermore, Li et al [5] extended GOP into speech attributes to detect mispronunciation of onset consonants in learners' Chinese by decision trees. By using speech attributes, they have shown the possibility to provide corrective feedback to improve pronunciation.…”

Improving Mispronunciation Detection for Non-Native Learners with Multisource Information and LSTM-Based Deep Models

“…Faced with the challenges of inconsistency in non-native phone-based labeling and imperfect acoustic modeling, our previous work [18,19] has investigated articulatory-based modeling for CAPT, where speech attributes [20,21] {lee.wei, chl}@gatech.edu, marco.siniscalchi@unikore.it, nfychen@i2r.a-star.edu.sg…”