Mispronunciation Diagnosis of L2 English at Articulatory Level Using Articulatory Goodness-Of-Pronunciation Features

Abstract

This paper proposes a method to provide an articulatory diagnosis of English produced by Korean learners using articulatory Goodness-Of-Pronunciation (aGOP) features, which are based on the distinctive feature theory in phonology. Previous studies on mispronunciation diagnosis have mainly dealt with pronunciation errors at phone-level. They inform learners of which phone is recognized as a diagnosis, when the corresponding segment is realized as a mispronunciation. However, to provide learners more effective corrective feedback, diagnosis had better be performed at articulatory-level, such as place and manner of articulation, rather than at phone-level. This study aims to provide automatic articulatory diagnosis using articulationbased confidence scores. At first, the speech of learners is forced-aligned and recognized to compute the GOP and aGOPs. When the forced-aligned segment is a consonant, articulatory diagnosis is conducted in three articulatory categories: voicing, place of articulation, and manner of articulation. Otherwise, diagnosis is performed in terms of rounding, height, and backness corresponding to articulatory characteristics of vowels. Experimental results show that F1 scores for voicing, place, and manner corresponding to consonants are 0.828, 0.754, and 0.781, respectively, whereas F1 score for rounding, height, and backness corresponding to vowels are 0.843, 0.782, and 0.824, respectively. These results indicate that the proposed method yields effective articulatory diagnosis.

Pronunciation Scoring With Goodness of Pronunciation and Dynamic Time Warping

“…Sudhakara et al [16] introduced context-aware GOP which takes both senone and transition state probabilities into consideration. Ryu et al [17] inferred that pronunciation scoring must combine phone level as well as articulatorylevel diagnoses such as voicing, place of articulation, and manner of articulation on consonants. Lin et al [18] used the acoustic model and replaced the forced alignment layer with a self-attention layer to get an utterance score based on transfer learning, but the results greatly depend on fine-tuning the scores of datasets.…”

“…The former approaches discussed are based only on the pronunciation scoring based on the likelihood of individual phones in sequential order, hence is limited to phone-level features extraction of the audio file to test [13][14][15][16][17][18][19]. The latter checks for a minimum distance for comparing different length time series or MFCC or LPC for audio comparison [20], [21], [22].…”

Attention Model for Articulatory Features Detection

“…Another application of phones recognition and articulatory features estimation is Computer-Assisted Pronunciation Training. Some of the approaches are described in [8] and [9]. This paper dwells on applications of attention-based models to articulatory features detection.…”

“…Another application of phones recognition and articulatory features estimation is Computer-Assisted Pronunciation Training. Some of the approaches are described in [8] and [9].…”