Automatic Short Answer Grading System in Indonesian Language Using BERT Machine Learning

Wijaya, Marvin Chandra

doi:10.18280/ria.350609

Cited by 6 publications

(4 citation statements)

References 17 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study further used CNN, RNN, and BOW as classifiers to detect the correct answers. Wijaya et al [32] proposed a BERT model to automatically grade short answers. The model was trained by using the Indonesian language.…”

Section: Transformer Learning Modelsmentioning

confidence: 99%

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

et al. 2023

View full text Add to dashboard Cite

In natural language processing, short-text semantic similarity (STSS) is a very prominent field. It has a significant impact on a broad range of applications, such as question–answering systems, information retrieval, entity recognition, text analytics, sentiment classification, and so on. Despite their widespread use, many traditional machine learning techniques are incapable of identifying the semantics of short text. Traditional methods are based on ontologies, knowledge graphs, and corpus-based methods. The performance of these methods is influenced by the manually defined rules. Applying such measures is still difficult, since it poses various semantic challenges. In the existing literature, the most recent advances in short-text semantic similarity (STSS) research are not included. This study presents the systematic literature review (SLR) with the aim to (i) explain short sentence barriers in semantic similarity, (ii) identify the most appropriate standard deep learning techniques for the semantics of a short text, (iii) classify the language models that produce high-level contextual semantic information, (iv) determine appropriate datasets that are only intended for short text, and (v) highlight research challenges and proposed future improvements. To the best of our knowledge, we have provided an in-depth, comprehensive, and systematic review of short text semantic similarity trends, which will assist the researchers to reuse and enhance the semantic information.

show abstract

Section: Transformer Learning Modelsmentioning

confidence: 99%

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The authors evaluated the performance of their proposed model on several benchmark datasets and achieved state-of-the-art results in short answer selection tasks. Wijaya et al [20] leveraged the BERT model to devise an automated grading system for short answers in the Indonesian language. The study employed Cohen's Kappa to assess the inter-rater reliability among student answers, and the model demonstrated high accuracy in grading the short answers.…”

Section: Related Workmentioning

confidence: 99%

“…Luo et al [21] explored the use of the BERT model for grading short answers, similar to the study by Wijaya et al [20]. However, they utilized a different dataset for training the model, which was the short answer scoring V2.0 dataset.…”

Section: Related Workmentioning

confidence: 99%

Machine Learning Model for Automated Assessment of Short Subjective Answers

Amur,

Hooi,

Bhanbro

et al. 2023

IJACSA

View full text Add to dashboard Cite

Natural Language Processing (NLP) has recently gained significant attention; where, semantic similarity techniques are widely used in diverse applications, such as information retrieval, question-answering systems, and sentiment analysis. One promising area where NLP is being applied, is personalized learning, where assessment and adaptive tests are used to capture students' cognitive abilities. In this context, openended questions are commonly used in assessments due to their simplicity, but their effectiveness depends on the type of answer expected. To improve comprehension, it is essential to understand the underlying meaning of short text answers, which is challenging due to their length, lack of clarity, and structure. Researchers have proposed various approaches, including distributed semantics and vector space models, However, assessing short answers using these methods presents significant challenges, but machine learning methods, such as transformer models with multi-head attention, have emerged as advanced techniques for understanding and assessing the underlying meaning of answers. This paper proposes a transformer learning model that utilizes multi-head attention to identify and assess students' short answers to overcome these issues. Our approach improves the performance of assessing the assessments and outperforms current state-of-the-art techniques. We believe our model has the potential to revolutionize personalized learning and significantly contribute to improving student outcomes.

show abstract

“…The grading system for short answers poses inherent challenges mated multiple-choice grading systems. It is essential to thoroughly ex and variations in these answers to ensure accurate assessment [8]. Th natural language processing (NLP) and machine learning application terest among educators in creating exams comprising open-ended qu automatically evaluated for a large number of students [5].…”

Section: Introductionmentioning

confidence: 99%

Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance

Wijanto,

Yong

2024

Applied Sciences

View full text Add to dashboard Cite

Short-answer questions can encourage students to express their understanding. However, these answers can vary widely, leading to subjective assessments. Automatic short answer grading (ASAG) has become an important field of research. Recent studies have demonstrated a good performance using computationally expensive models. Additionally, available datasets are often unbalanced in terms of quantity. This research attempts to combine a simpler SentenceTransformers model with a balanced dataset, using prompt engineering in GPT to generate new sentences. Our recommended model also tries to fine-tune several hyperparameters to achieve optimal results. The research results show that the relatively small-sized all-distilroberta-v1 model can achieve a Pearson correlation value of 0.9586. The RMSE, F1-score, and accuracy score also provide better performances. This model is combined with the fine-tuning of hyperparameters, such as the use of gradient checkpointing, the split-size ratio for testing and training data, and the pre-processing steps. The best result is obtained when the new generated dataset from the GPT data augmentation is implemented. The newly generated dataset from GPT data augmentation achieves a cosine similarity score of 0.8 for the correct category. When applied to other datasets, our proposed method also shows an improved performance. Therefore, we conclude that a relatively small-sized model combined with the fine-tuning of the appropriate hyperparameters and a balanced dataset can provide performance results that surpass other models that require larger resources and are computationally expensive.

show abstract

Automatic Short Answer Grading System in Indonesian Language Using BERT Machine Learning

Cited by 6 publications

References 17 publications

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

Short-Text Semantic Similarity (STSS): Techniques, Challenges and Future Perspectives

Machine Learning Model for Automated Assessment of Short Subjective Answers

Combining Balancing Dataset and SentenceTransformers to Improve Short Answer Grading Performance

Contact Info

Product

Resources

About