This study aims at investigating the effects of discourse modes on assessing EFL learners written performances. A total of fifty raters judged sixty essays (30 narratives and 30 argumentative writing modes) written by third-year English students from the Faculty of Letters and Humanities. Raters not only scored the compositions but also justified their scores assignments based on written explanations. Raters rating behaviors were diagnosed based on a variety of quantitative and qualitative tools. Essay scores were analyzed based on the statistical model FACETS to measure raters severity and internal consistency, task difficulty, and the scale functioning across writing modes. Qualitative data (gathered from interviews and report forms) were also analyzed in order to examine which aspects of writing were deemed more important than others across task types. The analysis revealed that the discourse mode was substantially an influential factor. The narrative task was more difficult than the argumentative one. Narrative essays were judged harsher than argumentative essays. Less consistent ratings could be detected from the narrative mode, compared to the argumentative one. Qualitative findings showed that the two writing modes were different in their qualitative judgments due to their different genre requirements and norms.
This study examines the interaction effect between rater groups and tasks on evaluating EFL learners’ written performances. Fifty raters took part in this study. The experienced rater group (n=25) and the novice rater group (n=25) judged sixty essays (30 narratives and 30 argumentative writing modes) written by third-year English students. Raters’ decision-making behaviours, in terms of scores assignment and written comments, were diagnosed based on different quantitative and qualitative tools. Scores were analysed based on FACETS to examine the effects of rater-task interaction on raters’ severity and internal consistency and the analytic scale’s functionality. Qualitative data were also analysed to diagnose which aspects of writing were deemed more important than others across rater groups and task types. The analysis revealed that both raters and tasks were substantially influential factors. The majority of expert raters displayed more severity in assessing narrative essays than argumentative essays. Different qualitative judgments are also detected across raters and tasks due to rating experience and task requirements. The findings of this study reflected implications not only for testing learners’ writing proficiency but also for test validation research in the task-based writing performance assessment field.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.