Severity differences among self-assessors, peer-assessors, and teacher assessors rating EFL essays

Esfandiari, Rajab; Myford, Carol M.

doi:10.1016/j.asw.2012.12.002

Cited by 24 publications

(20 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On average, peer-assessors were reliable assessors, not showing any misfit, and they rated EFL essays as similarly as did teacher assessors. These findings support some of those from previous studies (Esfandiari & Myford, 2013;Chang, et al, 2011;Nakamura, 2002;Zhao & Gallant (2012). Peer-assessors, as a group in this study, seem to distinguish between conceptually distinct assessment criteria, avoiding assigning similar ratings to students across these criteria.…”

Section: Discussionsupporting

confidence: 92%

“…Although peer-assessors' ratings in this study were generally as close as possible to those of teacher assessors, individual peer-assessors showed varying degrees of both central tendency effect and halo effect. Having investigated severity differences among three assessor types, Esfandiari and Myford (2013) concluded that -it is still premature to suggest that teachers use peer-assessors' ratings when they are making summative judgments about the writing ability of students in their courses‖ (p. 127). This conclusion still seems to remain true as demonstrated in this study; admittedly, so few studies do examine peer-assessment as a measurement tool in language education (for the most recent discussion, see Li, et.…”

Section: Resultsmentioning

confidence: 99%

“…The scale categories were very poor (1), poor (2), fair (3), good (4), very good (5), and excellent (6). The researcher borrowed the scale from Esfandiari and Myford (2013). Since in the present study, typed essays were used for rating, assessment criterion 15 was changed from punctuation, capitalization, and handwriting to punctuation, capitalization, and typing with no difference in proper functioning of the scale.…”

Section: The Instrumentmentioning

confidence: 99%

See 2 more Smart Citations

Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model

Esfandiari¹

2015

IJAL

Self Cite

View full text Add to dashboard Cite

In this study, the researcher used the many-facet Rasch measurement model (MFRM) to detect two pervasive rater errors among peer-assessors rating EFL essays. The researcher also compared the ratings of peer-assessors to those of teacher assessors to gain a clearer understanding of the ratings of peer-assessors. To that end, the researcher used a fully crossed design in which all peer-assessors rated all the essays MA students enrolled in two Advanced Writing classes in two private universities in Iran wrote. The peer-assessors used a 6-point analytic rating scale to evaluate the essays on 15 assessment criteria. The results of Facets analyses showed that, as a group, peer-assessors did not show central tendency effect and halo effect; however, individual peer-assessors showed varying degrees of central tendency effect and halo effect. Further, the ratings of peer-assessors and those of teacher assessors were not statistically significantly different.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Resultsmentioning

confidence: 99%

Section: The Instrumentmentioning

confidence: 99%

See 1 more Smart Citation

Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model

Esfandiari¹

2015

IJAL

Self Cite

View full text Add to dashboard Cite

show abstract

“…Cultural factors, such as the pressure to display overt modesty, which is valued in Chinese culture, may make students more critical of their own performance (Chen, 2008;Oscarson, 1997). In contrast, Iranian students are lenient when rating themselves since overt or false modesty concerning one's accomplishments is not accentuated in their culture (Esfandiari & Myford, 2013). Young children tend to over-estimate due to their wishful thinking and lack of the cognitive skills to evaluate their abilities accurately (Ross, 2006).…”

Section: Functions Of Reinforcementmentioning

confidence: 99%

Relationships between Peer- and Self-Assessment and Teacher Assessment of Young EFL Learners’ Oral Presentations

Hung

Samuelson

Chen³

2016

Educational Linguistics

View full text Add to dashboard Cite

As the traditional grammar translation approach is being gradually replaced by communicative or task-based approaches, paper-and-pencil tests, commonly used in English classes in Taiwan, do not meet the course goals. Alternative assessment, known for increasing learners' cognitive and meta-cognitive development as well as empowering students to take ownership of their learning, has been practiced extensively in L1 higher education, but neglected in L2 elementary schools. Thus, the purpose of this study is to investigate how peer and self-assessment can be implemented to evaluate young EFL learners' oral presentation and how the students perceive this experience. The study was conducted in two sixth grade classes at a public elementary school in southern Taiwan. After attending a professional development workshop held by the government, a local English teacher practiced peer and self-assessment in her class so as to engage every student in class activities and also to provide an opportunity for them to reflect upon their performance. In the process, the students formed groups of six to discuss and give grades after each individual student's oral report. Three types of data sources were analyzed. The first was the evaluation rubrics from peer groups, each presenting students, and the teacher. Then, a survey, containing 16 closed-format questions and one open-ended question, was administered to elicit the students' perceptions of the assessment process. Also, an interview was done with the teacher. The results show that peer and teacher assessment had strong positive correlation, whereas self-and teacher assessment were moderately correlated. The strength of correlation also varied for each evaluation criterion. Though learners responded positively to the assessing experiences in the questionnaires, they expressed concern that some grades assigned by peers were not fair and a few group members dominated the grading process. The findings shed light on benefits of combining peer and self-assessment and suggest training should emphasize self-assessment, evaluation criteria related to content of the presentation, and students' social skills to work harmoniously in groups. Most of all, students' traditional way of learning should not be neglected.

show abstract

“…Raters also judge students' writing ability differently depending on their academic background and sex (Vann, Lorenz & Meyer, 1991), and the training received (Weigle, 1994). Studies such as Cumming (1990), Eckes (2008), Esfandiari & Myford (2013), González & Roux (2013), Lim (2011), Shi (2001), Shi, Wan, & Wen (2003) and Wiseman (2012), describe how distinct rater backgrounds influence (or do not influence) their rating behavior, actual scores and scoring procedures. Lim (2011), for instance, focused on experienced and inexperienced raters.…”

Section: Introductionmentioning

confidence: 99%

Assessing EFL university students' writing: A study of score reliability

González

Trejo

Roux

2017

REDIE

View full text Add to dashboard Cite

The assessment of English as a Foreign Language (EFL) writing is a complex activity that is subject to human judgment, which makes it difficult to achieve a fair, accurate and reliable assessment of student writing (Pearson, 2004: 117;Hamp-Lyons, 2003). This study reports on the variability that exists between the analytical grades that 11 Mexican EFL university teachers awarded to five writing samples. It describes the raters' views on writing assessment and their use of analytical scoring rubrics. Data obtained from the grades awarded to each paper, and a background questionnaire, suggested that great variability was found between grades, and raters differed in their levels of leniency and severity, suggesting that having similar backgrounds and using the same rubric are not enough to ensure rater reliability. Participants' perceptions were found to be similar in terms of the use of analytical rubrics.Palabras: EFL writing, EFL writing assessment, Rater reliability, Analytical scoring rubric. ResumenLa evaluación de la escritura en inglés como lengua extranjera (EFL) es un proceso que depende del juicio humano, por lo que es difícil de obtener evaluaciones justas, acertadas y confiables (Pearson, 2004, p. 117;Hamp-Lyons, 2003). Este estudio reporta la variabilidad existente entre las calificaciones analíticas que 11 docentes Mexicanos universitarios de EFL proporcionaron a cinco trabajos escritos. Describe las percepciones de los participantes en torno a la evaluación de la escritura y el uso de las rúbricas de evaluación. Los datos obtenidos de las calificaciones de cada trabajo y de un cuestionario escrito revelaron que existe gran variedad entre las calificaciones proporcionadas y que los evaluadores difieren en sus niveles de exigencia, sugiriendo así que antecedentes homogéneos y el uso de una misma rúbrica 1 The authors presented preliminary results of this study at the 2014 TESL Canada Conference at the University of Regina, Saskatchewan, Canada, May 8-10, 2014 Assessing EFL university students' writing: a study of score reliability Palabras clave: Inglés como lengua extranjera, Evaluación de la escritura, Confiabilidad, Rúbrica de evaluación.

show abstract

Severity differences among self-assessors, peer-assessors, and teacher assessors rating EFL essays

Cited by 24 publications

References 56 publications

Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model

Rater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model

Relationships between Peer- and Self-Assessment and Teacher Assessment of Young EFL Learners’ Oral Presentations

Assessing EFL university students' writing: A study of score reliability

Contact Info

Product

Resources

About