An Effective End-to-End Modeling Approach for Mispronunciation Detection

Lo, Tien-Hong; Weng, Shi-Yan; Chang, Hsiu-Jui; Chen, Berlin

doi:10.48550/arxiv.2005.08440

Cited by 2 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, they trained an SVM classifier for each phoneme for mispronunciation detection to improve the system's ability to discriminate pronunciation quality [3]. Studies such as [17], [18], [19], [20] perform mispronunciation detection in terms of phonemes by building speech recognition models at the phoneme level. Caifeng Shen [4] and Ru Zhang [5] proposed an algorithm for Chinese Mandarin tone evaluation based on the improved Fujisaki model according to Chinese Mandarin tone pronunciation characteristics, respectively.…”

Section: Related Workmentioning

confidence: 99%

A Multifaceted Approach to Oral Assessment Based on the Conformer Architecture

Fan

Wumaier

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Automatic speaking assessment methods are essential for helping non-native learners to learn native pronunciation. The automated speaking assessment method consists of mispronunciation detection and pronunciation quality assessment. In the past, researchers have usually focused their research on only one specific aspect of the speaking assessment task. Research on multifaceted speaking tasks has been rare, and model building has often led to reduced performance due to the omission of local feature details. In this paper, we propose a multi-width band (MB) method and apply it to the Conformer model. This method can effectively increase the ability of the model to obtain local feature information at different scales. At the same time, we used a multi-task learning approach to train a multifaceted speaking assessment model based on GOP features. We conducted experiments on a self-built monosyllabic Mandarin mispronunciation detection dataset (PSC-MonoSyllable) and an English open-source pronunciation quality assessment dataset (SpeechOcean762), respectively. The experimental results show that the method's mispronunciation detection metrics in terms of phonemes, tones, and words on the PSC-MonoSyllable dataset (F1 scores) reached 70.18%, 80.06%, and 79.82%, respectively. The results of the method on the SpeechOcean 762 dataset for the pronunciation quality assessment task also showed a certain degree of improvement in all aspects of the phoneme-and grapheme-level correlation metrics compared with the baseline model.

show abstract