Diversity Among NEST Raters: How do New and Experienced NESTs Evaluate Korean English Learners’ Essays?

Lee, Kyoung Rang

doi:10.1007/s40299-016-0281-6

Cited by 3 publications

(9 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As Table 1 shows, the scores of the two groups were not significantly different, which is different from the writing evaluation. While the new group gave higher scores to the learners' essays than the experienced group in the previous study (Lee, 2016a) 6 , there was no difference in terms of speaking evaluation. What might have caused this change, whether the new group became more rigorous or the old group became more lenient to the learners' speaking, was further explored in terms of criteria.…”

Section: Differences In Generalcontrasting

confidence: 53%

“…Rater differences among NESTs as well as between NESTs and NNESTs have been explored in different contexts including Korea (Brown, 1991;Lee, 2009;Lee, 2016aLee, , 2016bMcNamara, 1996;Shin, 2010;Weigle, 1998;Winke, Gass, & Myford, 2012). Some researchers explored relationships between holistic scores and rating criteria (Shi, 2001) since analytic rubrics with several rating criteria would help raters more systematically than holistic rubrics and subjective holistic scoring (Hamp-Lyons, 1991;Kohn, 2006).…”

Section: Literature Reviewmentioning

confidence: 99%

“…If necessary, t-tests were also conducted as follow-up tests with an adjusted p-value in accordance with the number of pairs to be compared. The results of the previous study on writing (Lee, 2016a) were cited to be compared with those of this study on speaking.…”

Section: Data Analysis Proceduresmentioning

confidence: 99%

“…However, in a university where only NESTs teach and evaluate their students unlike middle or high schools 1 in Korea, variations within NESTs have been frequently observed. Based on the observation, it was examined whether their teaching experiences in one institute would cause any difference in evaluating Korean English learners' essays (Lee, 2016a). Interestingly, that study showed that those with more than five years of teaching experiences in Korea gave the highest scores to grammatically accurate essays, which was reported as NNESTs' rating behaviors rather than NESTs' in the previous studies (Brown, 1991;Lee, 2009), while the new ones gave the highest scores to the wellorganized essays.…”

Section: Introductionmentioning

confidence: 99%

“…If so, will experienced NESTs evaluate it differently from new NESTs as did they for writing? To answer the questions, this study aimed to explore any possible differences between new and experienced NESTs when evaluating Korean English learners' speaking, adopting the methodology of the previous study about the variations among NEST raters of writing (Lee, 2016a).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Different Rating Behaviors between New and Experienced NESTs When Evaluating Korean English Learners’ Speaking

Lee¹

2018

The Journal of AsiaTEFL

Self Cite

View full text Add to dashboard Cite

This study aimed to explore variations within native-English-speaking-teachers (NESTs) when evaluating English learners' speaking in terms of their teaching period in one institute (new and experienced) and to compare the results with their writing evaluation behaviors. Using the same methodology to the writing study, the speaking materials were carefully prepared and the speaking delivery performance like pronunciation were excluded to be compatible with the writing evaluation. While the experienced NESTs evaluated Korean English learners' essays more severely than the new NESTs in contrast to the previous studies, there was no difference in speaking. When evaluating the learners' English writing, the experienced group demonstrated similarities to non-native-Englishspeaking-teachers (NNESTs) in contrast to the new NESTs. When evaluating speaking, the new ones were more consistent in regarding content more substantially while the experienced ones showed a different rating behavior from writing. Lastly, unlike the previous studies, but like the writing study in comparison, the NESTs' perceived-difficulty played a more important role in grading than their perceived-importance regarding grammar, content, and vocabulary. The details of the results were elaborated and the discussions with implications were provided. This study also suggests a new approach to examine rater biases in relation to English learners' expressive skills.

show abstract

Section: Differences In Generalcontrasting

confidence: 53%

Section: Literature Reviewmentioning

confidence: 99%

Section: Data Analysis Proceduresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Different Rating Behaviors between New and Experienced NESTs When Evaluating Korean English Learners’ Speaking

Lee¹

2018

The Journal of AsiaTEFL

Self Cite

View full text Add to dashboard Cite

show abstract

An investigation into native and non-native teachers’ assessment of Chinese students’ English writing

Rao

Liu

2020

Southern African Linguistics and Applied Language Studies

View full text Add to dashboard Cite

Rater severity differences in English language as a second language speaking assessment based on rating experience, training experience, and teaching experience through many-faceted Rasch measurement analysis

Noh¹,

Matore

2022

Front. Psychol.

View full text Add to dashboard Cite

Evaluating candidates’ answers in speaking skill is difficult and rarely explored. This task is challenging and can bring inconsistency in the rating quality among raters, especially in speaking assessments. Severe raters will bring more harm than good to the results that candidates receive. Many-faceted Rasch measurement (MFRM) was used to explore the differences in teachers’ rating severity based on their rating experience, training experience, and teaching experience. The research uses a quantitative approach and a survey method to enlist 164 English teachers who teach lower secondary school pupils, who were chosen through a multistage clustered sampling procedure. All the facets involving teachers, candidates, items, and domains were calibrated using MFRM. Every teacher scored six candidates’ responses in a speaking test consisting of three question items, and they were evaluated across three domains, namely vocabulary, grammar, and communicative competence. Results highlight that the rating quality was different in terms of teachers’ rating experience and teaching experience. However, training experience did not bring any difference to teachers’ rating quality on speaking test. The evidence from this study suggests that the two main factors of teaching and rating experience must be considered when appointing raters for the speaking test. The quality of training must be improved to produce a rater with good professional judgment. Raters need to be supplied with answer samples with varied levels of candidates’ performance to practice before becoming a good rater. Further research might explore any other rater bias that may impact the psychological well-being of certain groups of students.

show abstract

Diversity Among NEST Raters: How do New and Experienced NESTs Evaluate Korean English Learners’ Essays?

Cited by 3 publications

References 17 publications

Different Rating Behaviors between New and Experienced NESTs When Evaluating Korean English Learners’ Speaking

Different Rating Behaviors between New and Experienced NESTs When Evaluating Korean English Learners’ Speaking

An investigation into native and non-native teachers’ assessment of Chinese students’ English writing

Rater severity differences in English language as a second language speaking assessment based on rating experience, training experience, and teaching experience through many-faceted Rasch measurement analysis

Contact Info

Product

Resources

About