A hierarchical classification approach to automated essay scoring

McNamara, Danielle S.; Crossley, Scott A.; Roscoe, Rod D.; Allen, Laura K.; Dai, Jie

doi:10.1016/j.asw.2014.09.002

Cited by 186 publications

(66 citation statements)

References 57 publications

(84 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, a number of studies have reported that linguistic properties related to local cohesion (i.e., cohesion between sentence level units; Halliday & Hasan, 1976) are either unrelated or negatively related to essay quality Crossley, Weston, McLain-Sullivan, & McNamara, 2011). Conversely, linguistic properties related to global cohesion (i.e., cohesion between larger chunks of texts such as paragraphs; Givón, 1995;Kintsch, 1995;Louwerse, 2005) have shown positive relations with essay quality (Crossley, Roscoe, McNamara, & Graesser, 2011;McNamara et al, 2013;McNamara, Crossley, Roscoe, & Dai, 2015). Hence, in terms of understanding essay quality, the results emerging from analyses of text cohesion properties have been inconclusive.…”

Section: Introductionmentioning

confidence: 99%

Say more and be more coherent: How text elaboration and cohesion can increase writing quality

Crossley¹,

McNamara²

2016

Journal of Writing Research

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Say more and be more coherent: How text elaboration and cohesion can increase writing quality

Crossley¹,

McNamara²

2016

Journal of Writing Research

View full text Add to dashboard Cite

“…Works that fall into this approach include (Srihari et al, 2008(Srihari et al, , 2007Cummins et al, 2016;McNamaraa et al, 2015). Further, authors in (Dong and Zhang, 2016) presented an empirical analysis of features typically used for learning AES models.…”

Section: Related Workmentioning

confidence: 99%

Automated Essay Scoring in the Presence of Biased Ratings

Amorim¹,

Cançado²,

Veloso³

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

View full text Add to dashboard Cite

Studies in Social Sciences have revealed that when people evaluate someone else, their evaluations often reflect their biases. As a result, rater bias may introduce highly subjective factors that make their evaluations inaccurate. This may affect automated essay scoring models in many ways, as these models are typically designed to model (potentially biased) essay raters. While there is sizeable literature on rater effects in general settings, it remains unknown how rater bias affects automated essay scoring. To this end, we present a new annotated corpus containing essays and their respective scores. Different from existing corpora, our corpus also contains comments provided by the raters in order to ground their scores. We present features to quantify rater bias based on their comments, and we found that rater bias plays an important role in automated essay scoring. We investigated the extent to which rater bias affects models based on hand-crafted features. Finally, we propose to rectify the training set by removing essays associated with potentially biased scores while learning the scoring model.

show abstract

“…E-Rater [14] is one of the first automated systems to evaluate text difficulty based on three general classes of essay features: structure (e.g., sentence syntax, proportion of spelling, grammar, usage or mechanics errors), organization based on various discourse features, and content based on prompt-specific vocabulary. Several other tools for automated essay grading or for assessing the textual complexity of a given text have been developed and employed in various educational programs [5,15]: Lexile (MetaMetrics), ATOS (Renaissance Learning), Degrees of Reading Power: DRP Analyzer (Questar Assessment, Inc.), REAP (Carnegie Mellon University), SourceRater (Educational Testing Service), Coh-Metrix (University of Memphis), Markit (Curtin University of Technology) [16], IntelliMetric [17] or Writing Pal (Arizona State University) [18,19].…”

Section: State Of the Artmentioning

confidence: 99%

ReaderBench Learns Dutch: Building a Comprehensive Automated Essay Scoring System for Dutch Language

Dascălu

Westera

Ruşeţi

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Automated Essay Scoring has gained a wider applicability and usage with the integration of advanced Natural Language Processing techniques which enabled in-depth analyses of discourse in order capture the specificities of written texts. In this paper, we introduce a novel Automatic Essay Scoring method for Dutch language, built within the Readerbench framework, which encompasses a wide range of textual complexity indices, as well as an automated segmentation approach. Our method was evaluated on a corpus of 173 technical reports automatically split into sections and subsections, thus forming a hierarchical structure on which textual complexity indices were subsequently applied. The stepwise regression model explained 30.5% of the variance in students' scores, while a Discriminant Function Analysis predicted with substantial accuracy (75.1%) whether they are high or low performance students.

show abstract

A hierarchical classification approach to automated essay scoring

Cited by 186 publications

References 57 publications

Say more and be more coherent: How text elaboration and cohesion can increase writing quality

Say more and be more coherent: How text elaboration and cohesion can increase writing quality

Automated Essay Scoring in the Presence of Biased Ratings

ReaderBench Learns Dutch: Building a Comprehensive Automated Essay Scoring System for Dutch Language

Contact Info

Product

Resources

About