A Multimodal Dialog System for Language Assessment: Current State and Future Directions

Abstract

We present work in progress on a multimodal dialog system for English language assessment using a modular cloud‐based architecture adhering to open industry standards. Among the modules being developed for the system, multiple modules heavily exploit machine learning techniques, including speech recognition, spoken language proficiency rating, speaker recognition, and the scoring of behaviors in multimodal data streams.

What scores from monologic speaking tests can(not) tell us about interactional competence

“…However, revising speaking tests to involve dialogic measures is likely to pose expensive logistical challenges and negatively impact practicality by its higher cost, in addition to the difficulty of balancing the need for authentic interaction on the one hand and standardization and fairness on the other. A more practical solution might be computer-based avatars, which are being developed (Ramanarayanan et al, 2020; Suendermann-Oeft et al, 2017), but it is uncertain to what extent they can really simulate interactions beyond very structured service encounters.…”

Scoring Interactional Aspects of Human-Machine Dialog for Language Learning and Assessment using Text Features

“…Conversational proficiency is a crucial skill for success in today's workplace (Weldy and Icenogle, 1997;Oliveri and Tannenbaum, 2019), which makes R&D on technologies that help develop and assess this skill important to complement our understanding from sociolinguistics (see for example Young, 2011;Doehler and Pochon-Berger, 2015). Dialog system technologies are one solution capable of addressing and automating this need by allowing learners to practice and improve their interactional compentence at scale (Suendermann-Oeft et al, 2017;Yu et al, 2019). However, such conversational technologies need to be able to provide targeted and actionable feedback to users in order for them to be useful to learners and widely adopted.…”