Landmark of Mandarin nasal codas and its application in pronunciation error detection

Xie, Yanlu; Hasegawa‐Johnson, Mark; Qu, Leyuan; Zhang, Jinsong

doi:10.1109/icassp.2016.7472703

Cited by 11 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, current attribute-level pronunciation score is just calculated using simple summation of frame-level posteriors. In the future, advanced method such as dynamic programing [35] and articulatory landmark mechanism [36], will also be studied. Finally, due to the limited page size, this preliminary work only reports the average mispronunciation detection performance and analyzes only a few trees in Figures 2-4, more individual phone-dependent detection performance will be analyzed in the near future with a full report.…”

Section: Resultsmentioning

confidence: 99%

Detecting Mispronunciations of L2 Learners and Providing Corrective Feedback Using Knowledge-Guided and Data-Driven Decision Trees

Siniscalchi

et al. 2016

Interspeech 2016

View full text Add to dashboard Cite

We propose a novel decision tree based framework to detect phonetic mispronunciations produced by L2 learners caused by using inaccurate speech attributes, such as manner and place of articulation. Compared with conventional score-based CAPT (computer assisted pronunciation training) systems, our proposed framework has three advantages: (1) each mispronunciation in a tree can be interpreted and communicated to the L2 learners by traversing the corresponding path from a leaf node to the root node; (2) corrective feedback based on speech attribute features, which are directly used to describe how consonants and vowels are produced using related articulators, can be provided to the L2 learners; and (3) by building the phone-dependent decision tree, the relative importance of the speech attribute features of a target phone can be automatically learned and used to distinguish itself from other phones. This information can provide L2 learners speech attribute feedback that is ranked in order of importance. In addition to the abovementioned advantages, experimental results confirm that the proposed approach can detect most pronunciation errors and provide accurate diagnostic feedback.

show abstract

Section: Resultsmentioning

confidence: 99%

Detecting Mispronunciations of L2 Learners and Providing Corrective Feedback Using Knowledge-Guided and Data-Driven Decision Trees

Siniscalchi

et al. 2016

Interspeech 2016

View full text Add to dashboard Cite

show abstract

“…Considering the purpose of CAPT, DA and FRR are more important in measuring the detection performance than FAR [2,27]. The evaluation models and the decision threshold were optimized by aiming at maximizing DA.…”

Section: Figure 2 : Roc Curves Of Evaluations On Held-out Test Set Fomentioning

confidence: 99%

Automatic Pronunciation Evaluation of Non-Native Mandarin Tone by Using Multi-Level Confidence Measures

2016

Self Cite

View full text Add to dashboard Cite

Automatic evaluation of tonal production plays an important role in a tonal language Computer-Assisted Pronunciation Training (CAPT) system. In this paper, we propose an automatic evaluation method for non-native Mandarin tones. The method applied multi-level confidence measures generated from Deep Neural Network (DNN). The confidence measures consisted of Log Posterior Ratios (LPR), Average Frame-level Log Posteriors (AFLP) and Segment-level Log Posteriors (SLP). The LPR was calculated between the correct tone model and competing tone models. The AFLP and LPR were obtained from frame-level scores. And the SLP was directly derived from segment-level scores. The multi-level confidence measures were modeled with a support vector machine (SVM) classifier. For comparison, three experiments were conducted according to different features: AFLP+LPR, SLP only and AFLP+LPR+SLP. The experimental results showed that the performance of the system which used multilevel confidence measures was the best, achieving a FRR of 5.63% and a DA of 82.45%, which demonstrated the efficiency of the proposed method.

show abstract

“…Indirect measurement models with CNN have received extensive attention over the last decades; Anjit and Rishidas [ 29 ] show the facial recognition model composing with two-layer deep CNN for feature extraction and SRC for classification. A variety of machine learning techniques have been used to predict nasal problems, such as the back-propagation neural network (BPNN) for the identification of the nasal bone [ 30 ], the long short-term memory neural network (LSTM) in SAHS event detection based on breathing [ 31 ], the random forest (RF) [ 32 ], and the support vector machine (SVM) [ 33 ]. Hybridization approaches integrating different machine learning techniques [ 34 ] have also been explored in recent years to improve nasal bone morphological prediction reliability and accuracy, with satisfactory morphology results.…”

Section: Introductionmentioning

confidence: 99%

Shape Prediction of Nasal Bones by Digital 2D-Photogrammetry of the Nose Based on Convolution and Back-Propagation Neural Network

Tuan

Hai

Thinh

2022

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

In rhinoplasty, it is necessary to consider the correlation between the anthropometric indicators of the nasal bone, so that it prevents surgical complications and enhances the patient’s satisfaction. The penetrating form of high-energy electromagnetic radiation is highly impacted on human health, which has often raised concerns of alternative method for facial analysis. The critical stage to assess nasal morphology is the nasal analysis on its anthropology that is highly reliant on the understanding of the structural features of the nasal radix. For example, the shape and size of nasal bone features, skin thickness, and also body factors aggregated from different facial anthropology values. In medical diagnosis, however, the morphology of the nasal bone is determined manually and significantly relies on the clinician’s expertise. Furthermore, the evaluation anthropological keypoint of the nasal bone is nonrepeatable and laborious, also finding widely differ and intralaboratory variability in the results because of facial soft tissue and equipment defects. In order to overcome these problems, we propose specialized convolutional neural network (CNN) architecture to accurately predict nasal measurement based on digital 2D photogrammetry. To boost performance and efficacy, it is deliberately constructed with many layers and different filter sizes, with less filters and optimizing parameters. Through its result, the back-propagation neural network (BPNN) indicated the correlation between differences in human body factors mentioned are height, weight known as body mass index (BMI), age, gender, and the nasal bone dimension of the participant. With full of parameters could the nasal morphology be diagnostic continuously. The model’s performance is evaluated on various newest architecture models such as DenseNet, ConvNet, Inception, VGG, and MobileNet. Experiments were directly conducted on different facials. The results show the proposed architecture worked well in terms of nasal properties achieved which utilize four statistical criteria named mean average precision (mAP), mean absolute error (MAE), R -square ( R 2 ), and T -test analyzed. Data has also shown that the nasal shape of Southeast Asians, especially Vietnamese, could be divided into different types in two perspective views. From cadavers for bony datasets, nasal bones can be classified into 2 morphological types in the lateral view which “V” shape was presented by 78.8% and the remains were “S” shape evaluated based on Lazovic (2015). With 2 angular dimension averages are 136.41 ± 7.99 and 104.25 ± 5.95 represented by the nasofrontal angle (g-n-prn) and the nasomental angle (n-prn-sn), respectively. For frontal view, classified by Hwang, Tae-Sun, et al. (2005), nasal morphology of Vietnamese participants could be divided into three types: type A was present in 57.6% and type B was present in 30.3% of the noses. In particular, types C, D, and E were not a common form of Vietnamese which includes the remaining number of participants. In conclusion, the proposed model performed the potential hybrid of CNN and BPNN with its application to give expected accuracy in terms of keypoint localization and nasal morphology regression. Nasal analysis can replace MRI imaging diagnostics that are reflected by the risk to human body.

show abstract

Landmark of Mandarin nasal codas and its application in pronunciation error detection

Cited by 11 publications

References 14 publications

Detecting Mispronunciations of L2 Learners and Providing Corrective Feedback Using Knowledge-Guided and Data-Driven Decision Trees

Detecting Mispronunciations of L2 Learners and Providing Corrective Feedback Using Knowledge-Guided and Data-Driven Decision Trees

Automatic Pronunciation Evaluation of Non-Native Mandarin Tone by Using Multi-Level Confidence Measures

Shape Prediction of Nasal Bones by Digital 2D-Photogrammetry of the Nose Based on Convolution and Back-Propagation Neural Network

Contact Info

Product

Resources

About