A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

Abstract

Recent years have seen increased interests in applying the latest technological innovations, including artificial intelligence (AI) and machine learning (ML), to the field of education. One of the main areas of interest to researchers is the use of ML to assist teachers in assessing students’ work on the one hand and to promote effective self-tutoring on the other hand. In this paper, we present a survey of the latest ML approaches to the automated evaluation of students’ natural language free-text, including both short answers to questions and full essays. Existing systematic literature reviews on the subject often emphasise an exhaustive and methodical study selection process and do not provide much detail on individual studies or a technical background to the task. In contrast, we present an accessible survey of the current state-of-the-art in student free-text evaluation and target a wider audience that is not necessarily familiar with the task or with ML-based text analysis in natural language processing (NLP). We motivate and contextualise the task from an application perspective, illustrate popular feature-based and neural model architectures and present a selection of the latest work in the area. We also remark on trends and challenges in the field.

Neural Networks or Linguistic Features? - Comparing Different Machine-Learning Approaches for Automated Assessment of Text Quality Traits Among L1- and L2-Learners’ Argumentative Essays

“…Despite the non-signi cant results, the hybrid consistently proved to perform better than the single-resource models across prompts and traits. This nding meets expectations and is in line with recent ndings from holistic scoring (Bai & Stede, 2022;. Furthermore, it implies that both types of input indeed capture partially different text information relevant for scoring essay traits.…”

“…The implementation of such attention mechanisms in large pre-trained transformer models has recently led to signi cant improvements and breakthroughs in various NLP tasks. In AES, the application of transformer models has also led to state-of-the-art performances (Bai & Stede, 2022;Wang et al, 2022;Xue et al, 2021). On the one hand, DNN models provide a powerful approach to AES with no need for elaborated feature engineering and with the promise of capturing content much better than n-gram or other content feature approaches such as prompt-similarity analysis or topic dictionaries.…”

“…Assessing students' free-text answers (e.g., argumentative essays) is an important task for arti cial intelligence (AI) and natural language processing (NLP) in education. This also involves developing tutoring systems based on AI-driven assessment procedures (e.g., Bai & Stede, 2022;Mathias & Bhattacharyya, 2020). Advantages of such systems that are mentioned in existing research involve (1) reduced workload for teachers, (2) immediate information about the performance level of their students without extensive manual correction effort, (3) more frequent and instant feedback for students, and (4) a consistent assessment procedure that is, for instance, not bound to human attention processes (see, e.g., Ramesh & Sanampudi, 2022;Uto, 2021;Yan, 2020).…”

“…Recent research has emphasized the promise of AI-based tutoring systems in supporting students closely during writing and learning processes (e.g., Hussein et al, 2019;Injadat et al, 2021). However, to support students in the context of complex writing tasks such as argumentative essays, an accurate and comprehensive assessment of several aspects of writing is necessary (Bai & Stede, 2022). Such negrained scoring of different aspects is a challenging problem that is largely unresolved in the eld of automated essay scoring (AES) (see, Horbach et al, 2017;Kumar & Boulanger, 2021).…”

Evaluation of Transformer-Based Neural Language Models for Writing Feedback and Automated Essay Scoring

“…Several studies have shown the advancements in natural language processing (NLP) and transformer-based neural language model have significantly influenced the development and effectiveness of automated essay scoring (AES) systems [6,9,13,15,18,[30][31][32][33][34][35][36][37][38].…”