Computer Grading of Student Prose, Using Modern Concepts and Software

Page, Ellis B.

doi:10.1080/00220973.1994.9943835

Cited by 168 publications

(96 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…6 AES scoring has a long history within writing assessment, dating back to the seminal work of Page (1966). More recently, several approaches to AES have come into use for scoring large-scale assessments, including the PEG system, a descendant of Page's original system (Page, 1994;1995), methods based on Latent Semantic Analysis (LSA; Foltz, Laham, & Landauer, 1999;Landauer, Laham, & Foltz, 2000), and the e-rater ® system (Attali & Burstein, 2006;Burstein et al, 1998 One branch of this tradition can be found in the literature on second language development in writing. Wolf-Quintero et al (1998) provided an excellent summary of this literature by discussing existing measures focused primarily on measurements of fluency, lexical and syntactic complexity, and accuracy in writing using such features as clause length and T-unit length, the normalized number of word types, the ratio of subordinate clauses to clauses, and the percent of text without error.…”

Section: • Collaboration Fosteredmentioning

confidence: 99%

Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency

Deane

Quinlan

Kostin

2011

ETS Research Report Series

View full text Add to dashboard Cite

ETS has recently instituted the Cognitively Based Assessments of, for, and as Learning (CBAL) research initiative to create a new generation of assessment designed from the ground up to enhance learning. It is intended as a general approach, covering multiple subject areas including reading, writing, and math. This paper is concerned with the writing assessment being developed within the CBAL framework, and in particular, with the potential for using automated scoring techniques effectively within such an assessment to support learning.Key words: writing assessment, CBAL, automated scoring ii (2008) Context and PurposeIn practice, assessment design involves complex tradeoffs. For instance, many testing programs assess writing directly by administering generic prompts under strict time limits. This type of assessment has gained in popularity relative to indirect measures of writing skill in response to the concern that writing should be assessed by actually requiring students to write, but it has been criticized as encouraging simplistic, formulaic approaches to instruction that fail to develop the complex skills students need to learn in order to become effective writers (Hillocks, 1987(Hillocks, , 2002. Portfolio-based writing assessment may encourage richer, more appropriate forms of instruction, but it raises issues of reliability and validity. Automated scoring methods have been introduced and advanced as instructional aids, but they have been criticized as focusing attention too much on mechanical correction of errors rather than encouraging critical engagement with content. In other words, it is no simple matter to design summative assessments that are truly tests worth teaching to (Shepard, 2002). Success in creating tests that truly encourage learning will require a complex, sustained effort that takes into account an extraordinary range of elements, including at least the following:• the cognitive and instructional literatures that define what students learn and how they best learn it• psychometric constraints on effective assessment• institutional and public policy constraints that ultimately determine test feasibility ETS has recently instituted a research initiative, Cognitively Based Assessments of, for, and as Learning (CBAL), intended to address this problem and create a new generation of assessment designed from the ground up to enhance learning. It is intended as a general approach, covering multiple subject areas including reading, writing, and math (see Bennett & Gitomer, 2009). This paper is concerned with the writing assessment being developed within the CBAL framework, and in particular, with the potential for using automated scoring techniques effectively within such an assessment to support learning. The role of automated scoring for CBAL cannot be discussed, however, without an in-depth presentation of the CBAL initiative.This presentation occupies section 1.2, after which section 1.3 examines the role of automated scoring within this philosophy. The remaining sections of th...

show abstract

Section: • Collaboration Fosteredmentioning

confidence: 99%

Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency

Deane

Quinlan

Kostin

2011

ETS Research Report Series

View full text Add to dashboard Cite

show abstract

“…The first version of e-rater (Burstein et al, 1998) used more than 60 features in the scoring process. PEG (Page, 1994) also uses dozens of mostly undisclosed features. One of the most important characteristics of e-rater V.2 is that it uses a small set of meaningful and intuitive features.…”

Section: The Feature Setmentioning

confidence: 99%

“…The first version of e-rater (Burstein et al, 1998) used a stepwise regression technique to select the best features that are most predictive for a given set of data. PEG (Page, 1994) is also based on regression analysis.…”

Section: Control and Judgment In Modelingmentioning

confidence: 99%

“…As early as 1966, Page showed that an automated "rater" is indistinguishable from human raters (Page, 1966). In the 1990's more systems were developed; the most prominent systems are the Intelligent Essay Assessor (Landauer, Foltz, & Laham, 1998), Intellimetric (Elliot, 2001), a new version of the Project Essay Grade (PEG, Page, 1994), and e-rater (Burstein et al, 1998).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automated Essay Scoring With E‐rater® V.2.0

Attali

Burstein

2004

ETS Research Report Series

399

545

View full text Add to dashboard Cite

Abstract:E-rater® has been used by the Educational Testing Service for automated essay scoring since 1999. This paper describes a new version of e-rater (V.2) that is different from other automated essay scoring systems in several important respects. The main innovations of e-rater V.2 are a small, intuitive, and meaningful set of features used for scoring; a single scoring model and standards can be used across all prompts of an assessment; modeling procedures that are transparent and flexible, and can be based entirely on expert judgment. The paper describes this new system and presents evidence on the validity and reliability of its scores.

show abstract

“…In view of this background, researchers including Page [15], Attali and Burstein [1], Burstein et al [6], and Foltz et al [11] have done a great deal of work on automated † † The author is with Tokyo University of Science, Yamaguchi, Sanyoonoda-shi, 756-0884 Japan.…”

Section: Became Univercity Student I Get Up Early Every Morning Imentioning

confidence: 99%

A Topic-Independent Method for Scoring Student Essay Content

Nagata

Kakegawa

Yabuta

2010

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper proposes a topic-independent method for automatically scoring essay content. Unlike conventional topic-dependent methods, it predicts the human-assigned score of a given essay without training essays written to the same topic as the target essay. To achieve this, this paper introduces a new measure called MIDF that measures how important and relevant a word is in a given essay. The proposed method predicts the score relying on the distribution of MIDF. Surprisingly, experiments show that the proposed method achieves an accuracy of 0.848 and performs as well as or even better than conventional topic-dependent methods. key words: essay scoring, language teaching and learning, student essay, essay topic, learner of English

show abstract

Computer Grading of Student Prose, Using Modern Concepts and Software

Cited by 168 publications

References 2 publications

Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency

Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency

Automated Essay Scoring With E‐rater® V.2.0

A Topic-Independent Method for Scoring Student Essay Content

Contact Info

Product

Resources

About