Self-Adaptive DNN for Improving Spoken Language Proficiency Assessment

Ye, Qian; Wang, Xinhao; Evanini, Keelan; Suendermann-Oeft, David

doi:10.21437/interspeech.2016-291

Cited by 19 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The baseline scoring model was built with approximately 130 automatic features extracted from the SpeechRater system, which can measure the pronunciation, prosody, fluency, rhythm, vocabulary, and grammar of spontaneous speech. All SpeechRater features were extracted either directly from the speech signal or from the output of a Kaldi-based automatic speech recognizer (Qian et al, 2016) with a word error rate of 20.9% on an independent evaluation set with non-native spontaneous speech from the TOEFL iBT speaking test.…”

Section: Methodsmentioning

confidence: 99%

“…The spread of English as the main global language for education and commerce is continuing, and there is a strong interest in developing assessment systems that can automatically score spontaneous speech from non-native speakers with the goals of reducing the burden on human raters, improving reliability, and generating feedback that can be used by language learners Higgins et al, 2011). Various features related to different aspects of speaking proficiency have been explored, such as features for pronunciation, prosody, and fluency (Cucchiarini et al, 2002;Chen et al, 2009;Cheng, 2011;Higgins et al, 2011), as well as features for vocabulary, grammar, and content Chen and Zechner, 2011;Chen and Zechner, 2011;Xie et al, 2012;Qian et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Rhetorical Structure Theory to Assess Discourse Coherence for Non-native Spontaneous Speech

Wang¹,

Gyawali²,

Bruno³

et al. 2019

Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019

Self Cite

View full text Add to dashboard Cite

This study aims to model the discourse structure of spontaneous spoken responses within the context of an assessment of English speaking proficiency for non-native speakers. Rhetorical Structure Theory (RST) has been commonly used in the analysis of discourse organization of written texts; however, limited research has been conducted to date on RST annotation and parsing of spoken language, in particular, non-native spontaneous speech. Due to the fact that the measurement of discourse coherence is typically a key metric in human scoring rubrics for assessments of spoken language, we conducted research to obtain RST annotations on non-native spoken responses from a standardized assessment of academic English proficiency. Subsequently, automatic parsers were trained on these annotations to process non-native spontaneous speech. Finally, a set of features were extracted from automatically generated RST trees to evaluate the discourse structure of nonnative spontaneous speech, which were then employed to further improve the validity of an automated speech scoring system.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Using Rhetorical Structure Theory to Assess Discourse Coherence for Non-native Spontaneous Speech

Wang¹,

Gyawali²,

Bruno³

et al. 2019

Proceedings of the Workshop on Discourse Relation Parsing and Treebanking 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…We trained a DNN-based ASR system using a fairly large corpus of non-native English adult speech, as described in Section 4.1, and made it self-adaptive to a test speaker in the children's corpus by i-vector-based speaker adaptation. This approach has been shown to be very effective for cross-task adaptation, from monologic speech to dialogic speech [27].…”

Section: A Speaker Adaption Of Dnn With I-vectorsmentioning

confidence: 99%

Improving DNN-Based Automatic Recognition of Non-native Children Speech with Adult Speech

Ye¹,

Wang²,

Evanini³

et al. 2016

5th Workshop on Child Computer Interaction (WOCCI 2016)

Self Cite

View full text Add to dashboard Cite

Acoustic models for state-of-the-art DNN-based speech recognition systems are typically trained using at least several hundred hours of task-specific training data. However, this amount of training data is not always available for some applications. In this paper, we investigate how to use an adult speech corpus to improve DNN-based automatic speech recognition for non-native children's speech. Although there are many acoustic and linguistic mismatches between the speech of adults and children, adult speech can still be used to boost the performance of a speech recognizer for children using acoustic modeling techniques based on the DNN framework. The experimental results show that the best recognition performance can be achieved by combining children's training data with adult training data of approximately the same size and initializing the DNN with the weights obtained by pre-training using the full training set of the adult corpus. This system can outperform the baseline system trained on only children's speech with an overall relative WER reduction of 11.9%. Among the three speaking tasks studied, the picture narration task shows the largest gain with a WER reduction from 24.6 % to 20.1%.

show abstract

“…Even with this mitigation, various papers have reported that improving ASR can lead to improvements in assessment with gains in machine-human correlation ranging from 0.02 to 0.7 e.g. [4,8,9]. Some features that may be beneficial for auto-marking are more affected by ASR errors, such as part-ofspeech tags and features related to the spoken content, so their use has been limited at present [9,10].…”

Section: Introductionmentioning

confidence: 99%

Impact of ASR Performance on Free Speaking Language Assessment

et al. 2018

View full text Add to dashboard Cite

In free speaking tests candidates respond in spontaneous speech to prompts. This form of test allows the spoken language proficiency of a non-native speaker of English to be assessed more fully than read aloud tests. As the candidate's responses are unscripted, transcription by automatic speech recognition (ASR) is essential for automated assessment. ASR will never be 100% accurate so any assessment system must seek to minimise and mitigate ASR errors. This paper considers the impact of ASR errors on the performance of free speaking test auto-marking systems. Firstly rich linguistically related features, based on part-of-speech tags from statistical parse trees, are investigated for assessment. Then, the impact of ASR errors on how well the system can detect whether a learner's answer is relevant to the question asked is evaluated. Finally, the impact that these errors may have on the ability of the system to provide detailed feedback to the learner is analysed. In particular, pronunciation and grammatical errors are considered as these are important in helping a learner to make progress. As feedback resulting from an ASR error would be highly confusing, an approach to mitigate this problem using confidence scores is also analysed.

show abstract

Self-Adaptive DNN for Improving Spoken Language Proficiency Assessment

Cited by 19 publications

References 11 publications

Using Rhetorical Structure Theory to Assess Discourse Coherence for Non-native Spontaneous Speech

Using Rhetorical Structure Theory to Assess Discourse Coherence for Non-native Spontaneous Speech

Improving DNN-Based Automatic Recognition of Non-native Children Speech with Adult Speech

Impact of ASR Performance on Free Speaking Language Assessment

Contact Info

Product

Resources

About