Multilingual Speech Evaluation: Case Studies on English, Malay and Tamil

Abstract

Speech evaluation is an essential component in computerassisted language learning (CALL). While speech evaluation on English has been popular, automatic speech scoring on low resource languages remains challenging. Work in this area has focused on monolingual specific designs and handcrafted features stemming from resource-rich languages like English. Such approaches are often difficult to generalize to other languages, especially if we also want to consider suprasegmental qualities such as rhythm. In this work, we examine three different languages that possess distinct rhythm patterns: English (stresstimed), Malay (syllable-timed), and Tamil (mora-timed). We exploit robust feature representations inspired by music processing and vector representation learning. Empirical validations show consistent gains for all three languages when predicting pronunciation, rhythm and intonation performance.

An ASR-free Fluency Scoring Approach with Self-Supervised Learning

“…In the features based approaches, various types of fluency related features, e.g. statistics of speech breaks [6][7][8], speech rate [6][7][8][9][10], filled pauses [7] and goodness of pronunciation (GOP) [11,12], are computed from time stamps provided for input speech, e.g., beginning and end time of words, phonemes. Different machine learning models, e.g., support vector machine (SVM) [7,13] and feed-forward neural networks [8], were trained to predict fluency scores from the extracted features.…”