Feature Engineering and Ensemble-Based Approach for Improving Automatic Short-Answer Grading Performance

Sahu, Archana; Bhowmick, Plaban Kumar

doi:10.1109/tlt.2019.2897997

Cited by 33 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among their conclusions is that automatic feedback is being used more in structured questions that require well-defined answers. These questions include multiple-choice [18], fill-in-the-blank [19], or those with a solution presented in a structured language, i.e., a mathematical formula [20] or a program [21,22]. The main positive effects of automatic feedback include the students using the feedback for improvement [23], increased student engagement [24,25], and reduction of instructor bias [26].…”

Section: Related Workmentioning

confidence: 99%

A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings

Hahn

Navarro

de-la-Fuente-Valentín

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Automatic scoring and feedback tools have become critical components of online learning proliferation. These tools range from multiple-choice questions to grading essays using machine learning (ML). Learning environments such as massive open online courses (MOOCs) would not be possible without them. The usage of this mechanism has brought many exciting areas of study, from the design of questions to the ML grading tools' precision and accuracy. This paper analyzes the findings of 125 studies published in journals and proceedings between 2016 and 2020 on the usages of automatic scoring and feedback as a learning tool. This analysis gives an overview of the trends, challenges, and open questions in this research area. The results indicate that automatic scoring and feedback have many advantages. The most important benefits include enabling scaling the number of students without adding a proportional number of instructors, improving the student experience by reducing the time between submission grading and feedback, and removing bias in scoring.On the other hand, these technologies have some drawbacks. The main problem is creating a disincentive to develop innovative answers that do not match the expected one or have not been considered when preparing the problem. Another drawback is potentially training the student to answer the question instead of learning the concepts. With this, given the exitance of a correct answer, such an answer could be leaked to the internet, making it easier for students to avoid solving the problem. Overall, each of these drawbacks presents an opportunity to look at ways to improve technologies to use these tools to provide a better learning experience to students.

show abstract

Section: Related Workmentioning

confidence: 99%

A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings

Hahn

Navarro

de-la-Fuente-Valentín

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In general, there are two approaches used to automatically evaluate short answers. The first approach uses a supervised method (Roy et al, 2016;Sultan et al, 2016;Sahu and Bhowmick, 2020) that extracts features in the short answers. The second approach uses a variety of unsupervised methods to determine scores based on the distance between the learner responses and the answer model (Bin et al, 2008;Mohler and Mihalcea, 2009;Hasanah et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

“…The second approach uses a variety of unsupervised methods to determine scores based on the distance between the learner responses and the answer model (Bin et al, 2008;Mohler and Mihalcea, 2009;Hasanah et al, 2018). Sahu and Bhowmick (2020) conducted a comparative study of different features and regression models to improve ASAG. A set of text similarity features, such as knowledgebased measures, corpus-based features, and word-embedding features, were extracted for each pair of learner response and reference answer.…”

Section: Related Workmentioning

confidence: 99%

“…Automatic short-answer grading (ASAG) is the process of evaluating this type of question response through a computer program by matching it with a related reference model (Sahu and Bhowmick, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Semantic similarity between learner answers (LA) and reference answers (RA) is the focus of many kinds of research related to ASAG (Mohler and Mihalcea, 2009;Mohler et al, 2011;Luchoomun et al, 2019). Three approaches to determine semantic similarity are knowledge-based, corpus-based, and word-embedding-based measures (Gomaa and Fahmy, 2013;Sahu and Bhowmick, 2020). Corpus-based similarity measures determine how many words are alike according to information obtained from large corpora (Gomaa and Fahmy, 2013).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automated Short-Answer Grading using Semantic Similarity based on Word Embedding

Lubis¹,

Mutaqin²,

Putri³

et al. 2021

IJTech

View full text Add to dashboard Cite

Automatic short-answer grading (ASAG) is a system that aims to help speed up the assessment process without an instructor's intervention. Previous research had successfully built an ASAG system whose performance had a correlation of 0.66 and mean absolute error (MAE) starting from 0.94 with a conventionally graded set. However, this study had a weakness in the need for more than one reference answer for each question. It used a string-based equation method and keyword matching process to measure the sentences' similarity in order to produce an assessment rubric. Thus, our study aimed to build a more concise short-answer automatic scoring system using a single reference answer. The mechanism used a semantic similarity measurement approach through word embedding techniques and syntactic analysis to assess the learner's accuracy. Based on the experiment results, the semantic similarity approach showed a correlation value of 0.70 and an MAE of 0.70 when compared with the grading reference.

show abstract

Fooling Automatic Short Answer Grading Systems

Filighera

Steuer

Rensing

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Feature Engineering and Ensemble-Based Approach for Improving Automatic Short-Answer Grading Performance

Cited by 33 publications

References 28 publications

A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings

A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational Settings

Automated Short-Answer Grading using Semantic Similarity based on Word Embedding

Fooling Automatic Short Answer Grading Systems

Contact Info

Product

Resources

About