Assessing EFL university students' writing: A study of score reliability

Abstract

The assessment of English as a Foreign Language (EFL) writing is a complex activity that is subject to human judgment, which makes it difficult to achieve a fair, accurate and reliable assessment of student writing (Pearson, 2004: 117;Hamp-Lyons, 2003). This study reports on the variability that exists between the analytical grades that 11 Mexican EFL university teachers awarded to five writing samples. It describes the raters' views on writing assessment and their use of analytical scoring rubrics. Data obtained from the grades awarded to each paper, and a background questionnaire, suggested that great variability was found between grades, and raters differed in their levels of leniency and severity, suggesting that having similar backgrounds and using the same rubric are not enough to ensure rater reliability. Participants' perceptions were found to be similar in terms of the use of analytical rubrics.Palabras: EFL writing, EFL writing assessment, Rater reliability, Analytical scoring rubric. ResumenLa evaluación de la escritura en inglés como lengua extranjera (EFL) es un proceso que depende del juicio humano, por lo que es difícil de obtener evaluaciones justas, acertadas y confiables (Pearson, 2004, p. 117;Hamp-Lyons, 2003). Este estudio reporta la variabilidad existente entre las calificaciones analíticas que 11 docentes Mexicanos universitarios de EFL proporcionaron a cinco trabajos escritos. Describe las percepciones de los participantes en torno a la evaluación de la escritura y el uso de las rúbricas de evaluación. Los datos obtenidos de las calificaciones de cada trabajo y de un cuestionario escrito revelaron que existe gran variedad entre las calificaciones proporcionadas y que los evaluadores difieren en sus niveles de exigencia, sugiriendo así que antecedentes homogéneos y el uso de una misma rúbrica 1 The authors presented preliminary results of this study at the 2014 TESL Canada Conference at the University of Regina, Saskatchewan, Canada, May 8-10, 2014 Assessing EFL university students' writing: a study of score reliability Palabras clave: Inglés como lengua extranjera, Evaluación de la escritura, Confiabilidad, Rúbrica de evaluación.

Exploring the Use of ESL Composition Profile for College Writing in the Indonesian Context

“…Scoring students' composition is a demanding task, especially in English as a foreign language context (EFL). Writing assessment is a complex activity that depends heavily on an individual judgment which is both time and energy consuming (González, Trejo, & Roux, 2017). EFL writing teachers are often caught in endless checks of students' errors in grammar, punctuation, spelling, diction, cohesive devices, transitions between paragraphs, main ideas, and sentence connectors.…”

Feasibility Test of English Instructional Materials for Secretaries Through Moodle

“…Syllabus was a specification of what will be taught in a language program and the sequence that must be taught (Nunan, 2004). All syllabus assessments developed are based on the principles of syllabus preparation in IQF which include: Basis of development; Identity; Course Learning Outcome; Lesson Learning Outcome; Evaluation Indicator; Theme; Learning Experience; Evaluation Material; Material Design; Learning Method; Bibliography; Evaluation; and Time of Achievement (Nurwardani, et. al, 2016).…”