The Effects of Rating Experience and Discourse Modes on Assessing Efl Learners� Writing Performances

Abstract

This study aims at investigating the effects of discourse modes on assessing EFL learners written performances. A total of fifty raters judged sixty essays (30 narratives and 30 argumentative writing modes) written by third-year English students from the Faculty of Letters and Humanities. Raters not only scored the compositions but also justified their scores assignments based on written explanations. Raters rating behaviors were diagnosed based on a variety of quantitative and qualitative tools. Essay scores were analyzed based on the statistical model FACETS to measure raters severity and internal consistency, task difficulty, and the scale functioning across writing modes. Qualitative data (gathered from interviews and report forms) were also analyzed in order to examine which aspects of writing were deemed more important than others across task types. The analysis revealed that the discourse mode was substantially an influential factor. The narrative task was more difficult than the argumentative one. Narrative essays were judged harsher than argumentative essays. Less consistent ratings could be detected from the narrative mode, compared to the argumentative one. Qualitative findings showed that the two writing modes were different in their qualitative judgments due to their different genre requirements and norms.