Automatic Short Answer Grading With SemSpace Sense Vectors and MaLSTM

Abstract

Automatic assessment of exams is widely preferred by educators than multiple-choice exams because of its efficiency in measuring student performance, lack of subjectivity when evaluating student response, and faster evaluation time than the time consuming manual evaluation. In this study, a new approach for the Automatic Short Answer Grading (ASAG) is proposed using MaLSTM and the sense vectors obtained by SemSpace, a synset based sense embedding method built leveraging WordNet. Synset representations of the Student's answers and reference answers are given as input into parallel LSTM architecture, they are transformed into sentence representations in the hidden layer and the vectorial similarity of these two representation vectors are computed with Manhattan Similarity in the output layer. The proposed approach has been tested using the Mohler ASAG dataset and successful results are obtained in terms of Pearson (r) correlation and RMSE. Also, the proposed approach has been tested as a case study using a specific dataset (CU-NLP) created from the exam of the ''Natural Language Processing'' course in the Computer Engineering Department of Cukurova University. And it has achieved a successful correlation. The results obtained in the experiments show that the proposed system can be used efficiently and effectively in context-dependent ASAG tasks.

Domain-Specific Hybrid BERT based System for Automatic Short Answer Grading

“…Hence, similarity score results in a more robust grading due to better understanding of the key-response pair while reducing the biasness of base model towards higher scores. The Proposed model performs better than most of the existing models except MaLSTM model proposed by Tulu et al [10]. The reason being that their LSTM model is trained on each assignment separately rather than on the entire dataset in one go.…”

“…They used multi-domain resources as datasets to perform fine-tuning. Tulu et al [10] presented an ASAG system using sense vectors obtained from SemSpace algorithm and LSTM combined with Manhattan Vectorial Similarity. Sense embeddings of Synsets corresponding to each word in Student's answers or reference answers are given as input into parallel LSTM architecture.…”

Comparing Machine Learning and Human Judge in SATU Indonesia Awarding Processes

“…For lengthy text, more rigorous machine learning for text classification was developed [13][15]- [18]. Some researchers have gone so far as to make it more automated by using English Natural Language Processing (NLP) to assess short answers [11][13] [19]. The last technique may be challenging to do as the Indonesian NLP is not as mature as English [20] [21].…”

Short Answers Assessment Approach based on Semantic Network

“…Therefore, many researchers have proposed methods for evaluating answers or vignettes, which differ according to the techniques, used in text processing and score calculation. In [9], they proposed combining MaLSTM with sense vectors produced from SemSpace, a synchronization-based meaning fusion method that takes advantage of WordNet, to develop an approach to Automatic Classification of Short Answers (ASAG). As inputs to the parallel LSTM structure, simultaneous representations of student answers and reference answers are introduced, in the hidden layer.…”