An Automated Essay Scoring Based on Neural Networks to Predict and Classify Competence of Examinees in Community Academy

Abstract

AES has been widely used in assessing student learning outcomes. However, few studies use Automated Essay Scoring (AES) to simultaneously determine the community academy's competency test scores and levels. This study aims to apply AES to assess essays on the competency certification test. The AES can predict the examinees' scores and classify examinees' competency levels. The method used to build AES uses Back Propagation Neural Networks (BPNN). BPNN was chosen because of its simplicity and ease in building the model. The results showed that the AES for predicting the examinee's competency value showed the MAE value is 0.061621 and the accuracy value is = 97.9665 %. The results of the classification of student competency levels show Accuracy= 0.9063, Precision= 0.9167, Recall= 0.8888, and F1 Score= 0.8857.

Antithesis of Human Rater: Psychometric Responding to Shifts Competency Test Assessment Using Automation (AES System)

“…By comparing the results of the assessment tests on essay questions between automation and human raters, the problem frame can be seen whether there are significant differences, as the human rater can be labor-intensive and time-consuming (Zhang, 2013), whereas automation process brings otherwise results (Dong et al, 2017;Wong & Bong, 2019). By using the AES, statistical proof of the automation assessment results at the validity and reliability level becomes a benchmark, whether the consistency of the results supports the resulting accuracy value, or it still requires other proof from the human rater; the prediction results show the same/not much difference (Buditjahjanto et al, 2022). This phenomenon is still a topic of conversation in vocational education regarding the quality of outcomes of students at Akademi Komunitas Negeri (AKN) Pacitan and Blitar (research locus), because currently the competency test assessment still uses a human rater, and has an impact on recognizing ownership of a competency certificate as a competency in mastering knowledge, skills, and attitudes (Watkins, 2020).…”