Automatic Short Answer Grading via Multiway Attention Networks

Abstract

Automatic short answer grading (ASAG), which autonomously score student answers according to reference answers, provides a costeffective and consistent approach to teaching professionals and can reduce their monotonous and tedious grading workloads. However, ASAG is a very challenging task due to two reasons: (1) student answers are made up of free text which requires a deep semantic understanding; and (2) the questions are usually open-ended and across many domains in K-12 scenarios. In this paper, we propose a generalized end-to-end ASAG learning framework which aims to (1) autonomously extract linguistic information from both student and reference answers; and (2) accurately model the semantic relations between free-text student and reference answers in open-ended domain. The proposed ASAG model is evaluated on a large real-world K-12 dataset and can outperform the state-of-the-art baselines in terms of various evaluation metrics.

An AI-Based System for Formative and Summative Assessment in Data Science Courses

“…Such features have been further explored in Magooda et al (2016) to include a number of corpusbased, knowledge-based, word-based, and vector-based similarity indices. Recently, few studies have been presented using transformer-based architectures (Devlin et al 2019) and neural network classifiers to perform short-answer grading (Liu et al 2019;. Although neural approaches have demonstrated acceptable performance and generalisation capabilities across domains ) and languages (Camus and Filighera 2020), they typically require large amounts of data to learn an accurate classification model, as confirmed in the studies cited above.…”

“…We tested these models because they have achieved state-of-the-art performance in natural language inference tasks (Zhang et al 2018), which inspired the choice to concatenate the students' answers and correct answers in the proposed approach. They were also successfully applied to short-answer grading on English data, with larger datasets than ours (Liu et al 2019;. In our experiments, we adopted the Base Multilingual Cased model, 2 covering 104 languages with 12 layers, 768 dimensional states, 12 heads, and 110M parameters.…”

Automatic Essay Scoring: A Review on the Feature Analysis Techniques

“…The author in [28] proposed a multi-way attention architecture for AES task. The proposed architecture contains a transformer layer at first which process pre-trained Glove word embedding of student's answer and model's answer.…”

Recent Trends in Answer Script Evaluation – A Literature Survey

“…The NLP-based research works generally use semantic matching techniques for comparison of student answer with reference answer(s) for short answer grading. These works are found mostly effective for close-ended short answer questions [12][13][14][15][16].…”

“…Hence deep semantic understanding is required for a good ASAG system. The paper [15] proposes an ASAG framework for grading open-ended answers by modelling semantic relations between student answers and reference answers. The framework comprises of four layers.…”