Home-Grown Automated Essay Scoring in the Literature Classroom: A Solution for Managing the Crowd?

Abstract

Managing crowded classes in terms of classroom assessment is a difficult task due to the amount of time which needs to be devoted to providing feedback to student products. In this respect, the present study aimed to develop an automated essay scoring environment as a potential means to overcome this problem. Secondarily, the study aimed to test if automatically-given scores would correlate with the scores given by a human rater. A quantitative research design employing a machine learning approach was preferred to meet the aims of the study. The data set to be used for machine learning consisted of 160 scored literary analysis essays written in an English Literature course, each essay analyzing a theme in a given literary work. To train the automated scoring model, LightSide software was used. First, textual features were extracted and filtered. Then, Logistic Regression, SMO, SVO, Logistic Tree and Naïve Bayes text classification algorithms were tested by using 10-Fold Cross-Validation to reach the most accurate model. To see if the scores given by the computer correlated with the scores given by the human rater, Spearman's Rank Order Correlation Coefficient was calculated. The results showed that none of the algorithms were sufficiently accurate in terms of the scores of the essays within the data set. It was also seen that the scores given by the computer were not significantly correlated with the scores given by the human rater. The findings implied that the size of the data collected in an authentic classroom environment was too small for classification algorithms in terms of automated essay scoring for classroom assessment.

Towards Better Support for Machine-Assisted Human Grading of Short-Text Answers

“…Many publications discuss the potential threat such systems could cause when not configured well. Several publications [19,22,23,3,24,25] critically assess current methodological approaches and applicable tools. Conclusively, almost all of the mentioned publications emphasize that computer-based evaluation has strengths and weaknesses associated.…”

On the periphery of the parts of speech system

“…The classification of parts of speech is based on a number of criteria: semantic (the generalized lexical meaning), morphological (the combination of grammatical categories with the corresponding paradigms), and syntactic (functions of words in a sentence) (Valeeva, Aitov, Bulatbayeva, 2016;Sum, Kwon, 2018). The difference between autosemantic and synsemantic words and synsemantic groups of words also lies in the fact that the former form open classes of words and it is almost impossible to determine their quantitative composition (the exceptions are pronouns and numerals), whereas the latter can be regarded as closed or relatively closed classes (Geng, 2017;Uzun, 2018). The aim of the article is to consider the periphery of the parts of the speech system.…”

Stylistic means of headline creating in the information-analytical media text: pragmasemantic aspect of metaphor

“…For intention implementation, actors of a discourse (authors-journalists) update the estimated potential of language units, thereby complicating a reference with a connotation. Many researchers note the fact that depending on individual character of communicants and a situation of communication the same sign can cause the association preferable before other in representation of the person (Balli, 1955;Borev, 2005;Karpovskaya, 2015;Kolodeznikov, Kolodeznikova, 2016;Nikiforova, Ignatiev, 2016;Kamalova, Zakirova, 2017;Uzun, 2018;Sum, Kwon, 2018;Solmaz, 2018;Metruk, 2018;Simsek, Elciyar, Kizilhan, 2019, etc.). These associations are defined by individual, national, social, and other peculiarities of representatives of ethnoses and cultures (Karpovskaya, Tolstova, 2018, p. 105).…”