On the Use of Composition Scoring Techniques, Objective Measures, and Objective Tests to Evaluate ESL Writing Ability

Abstract

The evaluation of writing ability among both L1 and L2 students has become increasingly important in recent years because the results of such evaluation are used for a variety of administrative, instructional, and research purposes. Classroom teachers, in particular, because they want to use these results to help improve, influence, refine, and shape their students' attained writing ability, have specific concerns regarding the various methods of writing assessment available. Additional concerns, especially of teachers, include the issue of whether the results of any one evaluation procedure are helpful to students and how they affect students' writing performance and attitudes. This article does not address the question of how best to use the results of a writing evaluation to improve student writing, but it does present a discussion of the efficacy of various direct and indirect methods of assessing writing ability. The purpose of this article is to outline the assumptions, procedures, and consequences for use of the principal direct scoring schemes and objective measures presently available for evaluating compositions, and of the objective tests currently utilized to determine (indirectly) students' ability to write. It is hoped that ESL professionals (whether they are teachers, administrators, or researchers) can make use of this discussion as they try to decide how best to handle questions related to writing ability. Although some readers may question the appropriateness of using indirect measures, such as standardized tests, since those only measure recognition of surface‐level syntactic phenomena, the more widely used standardized tests have been included in this survey because they are used by a number of institutions as criterion measures as well as for placement and prediction.

Autoevaluación y Evaluación Docente en una Clase de Escritura de Inglés como Lengua Extranjera

“…According to Stiggins, Richard, Nancy, and Bridgeford (as cited in Perkins, 1983) "Holistic scoring calls for the reader to rate overall writing proficiency on a single rating scale, (but) analytic scoring breaks performance down into component parts (e.g. organization, wording, idea)" (p. 652).…”

“…Bacha (2001) confirmed that adopting either of these techniques depended on the purpose of writing in EFL programs. She maintained that to provide learners with more specific feedback, analytic scoring would be more appropriate since holistic scoring tends to be highly subjective and lacks internal consistency due to shifting standards (Perkins, 1983). Other scholars such as Hamp-Lyons (1995) have pointed out that "holistic scoring system is a closed system" (p. 760) and no one can have access to points for different parts since the raters do not have a certain criterion for scoring a piece of writing.…”

“…Cumming (1990), who discussed biases in holistic evaluations of ESL writings, claimed that "analytic scales may have the advantage of drawing raters' attention to specific aspects of students' composition" (p. 42). Studies have indicated that high interrater reliability has been obtained from analytic scoring (Bachman, 1990;Jacobs, Zingraf, Wormuth, Hartfiel, & Hughey, 1981;Perkins, 1983).Apart from having clear criteria for assessment, all raters need training in assessment. Lumley and McNamara (1995) argued that even teachers who wanted to act as raters needed some training courses to make them internally consistence or "self-consistent" (p. 57).…”

“…Cumming (1990), who discussed biases in holistic evaluations of ESL writings, claimed that "analytic scales may have the advantage of drawing raters' attention to specific aspects of students' composition" (p. 42). Studies have indicated that high interrater reliability has been obtained from analytic scoring (Bachman, 1990;Jacobs, Zingraf, Wormuth, Hartfiel, & Hughey, 1981;Perkins, 1983).…”

Assessing writing ability in primary education : on the evaluation of text quality and text complexity

“…Firstly, the raters may pay less attention to the written text as a whole by simply letting the questions lead them through the essay, causing text features to be isolated from the context (Perkins, 1983). Secondly, it is questionable whether the construct of writing ability can be 'captured' by a finite set of questions, meaning there is no guarantee that all relevant questions are being asked and, subsequently, that all relevant features of writing ability are assessed.…”