Portuguese Automatic Short Answer Grading

Abstract

Automatic Short Answer Grading is the study field that addresses the assessment of students' answers to questions in natural language. Besides length, it differs from automatic essay grading by focusing on the evaluation of content instead of answer's style. The grading of the answers is generally seen as a typical classification supervised learning. Many works have been recently developed, but most of them deal with data in the English language. In this paper, we present a new Portuguese dataset and system for automatic short answer grading. The data was collected with the participation of 13 teachers, 12 undergraduate students and 245 elementary school students. Results achieved 69% accuracy in four-class classification and 85% on binary classification.

The Influence of Variance in Learner Answers on Automatic Content Scoring

“…• The Portuguese PT_ASAG dataset (Galhardi et al, 2018) contains learner answers collected in biology classes at schools in Brazil using a web system. Apart from reference answers for each question, the dataset also contains keywords specifying aspects of a good question.…”

Automatic Grading of Portuguese Short Answers Using a Machine Learning Approach

“…With that done, a web system was modeled and developed in order to be operated by real world users [Galhardi and Brancher 2018a]. From the system's usage from teachers and students, a dataset was collected and packaged in order to be made publicly available on the web (Section 3) [Galhardi et al 2018].…”

“…Then, the different approaches were evaluated and compared. From them, a final model was created for grading new answers, composed of different groups of approaches (Section 6) [Galhardi et al 2018]. Finally, Section 7 presents the final considerations for this work.…”

AvaliaSmart: Avaliação Automática de Questões Discursivas

“…A quantidade escassa de trabalhos e a inexistência de um conjunto de dados maior na língua portuguesa levou à dúvida sobre a eficácia do modelo para o português também. Portanto, o próximo passo foi criar um conjunto de dados reais, a partir de alunos brasileiros do ensino fundamental e médio, adaptar e melhorar a ferramenta e realizar novos experimentos [Galhardi et al 2018a].…”