Using Articulatory Representations to Detect Segmental Errors in Nonnative Pronunciation

Narayanan, Shrikanth

doi:10.1109/tasl.2007.909330

Cited by 27 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The ASR based techniques benefit from their well-defined mathematical framework for GMM-HMM based acoustic modeling and many well-developed toolkits for computation. An alternative to ASR based approach is to use acoustic phonetic features as front-end and a classifier as a back-end such as [6], [7]. Within this framework, mispronunciation detection can be formulated more suitably as a classification problem and thus more discriminative features and classifiers can be explored.…”

Section: Introductionmentioning

confidence: 99%

Maximum F1-Score Discriminative Training Criterion for Automatic Mispronunciation Detection

Huang

Wang

et al. 2015

IEEE/ACM Trans. Audio Speech Lang. Process.

107

View full text Add to dashboard Cite

We carry out an in-depth investigation on a newly proposed Maximum F1-score Criterion (MFC) discriminative training objective function for Goodness of Pronunciation (GOP) based automatic mispronunciation detection that makes use of Gaussian Mixture Model-hidden Markov model (GMM-HMM) as acoustic models. The formulation of MFC seeks to directly optimize F1-score by converting the non-differentiable F1-score function into a continuous objective function to facilitate optimization. We present model-space training algorithm according to MFC using extended Baum-Welch form like update equations based on the weak-sense auxiliary function method. We then present MFC based feature-space discriminative training. We train a matrix projecting from posteriors of Gaussians to a normal size feature space, and add the projected features to traditional spectral features. Mispronunciation detection experiments show MFC based model-space training and feature-space training are effective in improving F1-score and other commonly used evaluation metrics. It is also shown MFC training in both the feature-space and model-space outperforms either model-space training or feature-space training alone, and is about 11.6% better than the maximum likelihood (ML) trained baseline in terms of F1-score. Further, we review and compare mispronunciation detection results with the use of MFC and some traditional training criteria that minimize word error rate in speech recognition. The experimental analysis and comparison provide useful insight into the correlations between F1-score maximization and optimization of these training criteria.

show abstract

Section: Introductionmentioning

confidence: 99%

Maximum F1-Score Discriminative Training Criterion for Automatic Mispronunciation Detection

Huang

Wang

et al. 2015

IEEE/ACM Trans. Audio Speech Lang. Process.

107

View full text Add to dashboard Cite

show abstract

“…Feature detection signifies that the analysis instead aims at finding aspects in the acoustic signal that can be translated into articulatory information, e.g. place and manner of articulation and lip rounding (Frankel Wester, & King, 2007;Teppermann & Narayanan, 2008). The output from each detector can then be compared to the features of the target and the feedback is based on the deviating features (Strik, Truong, de Wet, & Cucchiarini, 2009).…”

Section: Articulation Analysismentioning

confidence: 99%

Analysis of and feedback on phonetic features in pronunciation training with a virtual teacher

Engwall

2011

Computer Assisted Language Learning

View full text Add to dashboard Cite

Pronunciation errors may be caused by several different deviations from the target, such as voicing, intonation, insertions or deletions of segments, or that the articulators are placed incorrectly. Computer-animated pronunciation teachers could potentially provide important assistance on correcting all these types of deviations, but they have an additional benefit for articulatory errors. By making parts of the face transparent, they can show the correct position and shape of the tongue and provide audiovisual feedback on how to change erroneous articulations. Such a scenario however requires firstly that the learner's current articulation can be estimated with precision and secondly that the learner is able to imitate the articulatory changes suggested in the audiovisual feedback. This article discusses both these aspects, with one experiment on estimating the important articulatory features from a speaker through acoustic-to-articulatory inversion and one user test with a virtual pronunciation teacher, in which the articulatory changes made by seven learners who receive audiovisual feedback are monitored using ultrasound imaging.

show abstract

“…They cannot be represented by one symbol. So the phone set of 39 phones used in the CMU dictionary which is widely used in many speech systems [1] [2] is too small for speech recognition to represent all the phonemes in English. To improve the performance of our system, we decide to expand the phone set of the pronunciation dictionary using the knowledge of phonetics and observation of the spectral properties of some phonemes because this new phone set can express the phonemes in English more accurately.…”

Section: IIImentioning

confidence: 99%

Design of the pronunciation dictionary for an English CAPT system

Liu

Peng

et al. 2010

2010 International Conference on Computer Design and Applications

View full text Add to dashboard Cite

Computer Assisted Pronunciation Training (CAPT) systems can judge the overall pronunciation quality and point out the pronunciation errors by recognition of the user's utterance to improve the users' oral ability. However, the performance of the current error detection systems can't satisfy the users' expectation.This paper carefully designs the pronunciation dictionary of the speech recognition engine of the CAPT system based on observation of the spectral properties of some phonemes. First, the phonetic symbol lRi utilized in some modern English dictionaries is replaced by two symbols, lRi (as in word result) and IERi (as in word bear), due to the large spectral deviation in these two cases. Moreover, rather than using a consonant sequence to represent a consonant cluster, we use one symbol to represent the whole for some consonant clusters, according to the phonetic properties of the consonants. Finally, we divide the phoneme III into clear III and dark III according to the distinction in the manner of articulation and the place of articulation. So is In/. As a result, we get a new pronunciation dictionary which is more suitable for the speech recognition engine of the CAPT system. The HMMs are trained by using the TIMIT database, and evaluated on a database involving 40 Chinese undergraduates. Experimental results illustrate the effectiveness of the new pronunciation dictionary.

show abstract

Using Articulatory Representations to Detect Segmental Errors in Nonnative Pronunciation

Cited by 27 publications

References 20 publications

Maximum F1-Score Discriminative Training Criterion for Automatic Mispronunciation Detection

Maximum F1-Score Discriminative Training Criterion for Automatic Mispronunciation Detection

Analysis of and feedback on phonetic features in pronunciation training with a virtual teacher

Design of the pronunciation dictionary for an English CAPT system

Contact Info

Product

Resources

About