This study investigated human raters’ perceptions of automated assessment of oral language skills. The raters (n = 37) participated in three assessment rounds organized by the DigiTala research project using Moodle and Zoom. The raters assessed Finnish and Swedish learners’ speech samples using one holistic and five analytical rating scales created in the project. After the assessment, the raters responded to a questionnaire that included Likert-scale and open-ended questions. Numerical responses were analyzed with descriptive statistics, open responses with content analysis. The raters think that automated scoring could support human rating. The assessment rounds were carried out successfully. The selected dimensions proved to be essential parts of the speaking performances. The results will benefit those working on automated assessment and oral language assessment.
In this study, we interviewed four language teachers in order to investigate their perceptions of an automated tool developed for assessing Finnish and Swedish learners’ speech. We showed the first version of the Moodle plugin developed by the DigiTala research project to the teachers. In this version, after recording a speech sample, the system provides analytic and holistic feedback to the learner. The automated feedback was developed based on the rating criteria that human raters have used earlier in the project. The data consist of four semi-structured individual interviews conducted in Zoom. The interviews were transcribed and analysed using content analysis. We found that automated feedback has significant implication areas in language learning and teaching. Teachers found the feedback statements understandable and useful, even though they do not cover all dimensions relevant for the speaking construct. The results will benefit those working on automated assessment of oral language skills.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.