Different Rating Behaviors between New and Experienced NESTs When Evaluating Korean English Learners’ Speaking

Abstract

This study aimed to explore variations within native-English-speaking-teachers (NESTs) when evaluating English learners' speaking in terms of their teaching period in one institute (new and experienced) and to compare the results with their writing evaluation behaviors. Using the same methodology to the writing study, the speaking materials were carefully prepared and the speaking delivery performance like pronunciation were excluded to be compatible with the writing evaluation. While the experienced NESTs evaluated Korean English learners' essays more severely than the new NESTs in contrast to the previous studies, there was no difference in speaking. When evaluating the learners' English writing, the experienced group demonstrated similarities to non-native-Englishspeaking-teachers (NNESTs) in contrast to the new NESTs. When evaluating speaking, the new ones were more consistent in regarding content more substantially while the experienced ones showed a different rating behavior from writing. Lastly, unlike the previous studies, but like the writing study in comparison, the NESTs' perceived-difficulty played a more important role in grading than their perceived-importance regarding grammar, content, and vocabulary. The details of the results were elaborated and the discussions with implications were provided. This study also suggests a new approach to examine rater biases in relation to English learners' expressive skills.

Examining Rater Performance on the CELBAN Speaking: A Many-Facets Rasch Measurement Analysis

“…Lumley (2002) conducted MFRM analysis to assess rater performances in writing assessment and found that raters tend to rate severely on grammar. Caban (2003), Lee (2018), andMcNamara (1990) provided similar findings revealing that speaking raters interpreted each scoring category in a rather different way.…”