Rater attitude towards emerging varieties of English: a new rater effect?

Abstract

Background: A strong interest in researching World Englishes (WE) in relation to language assessment has become an emerging theme in language assessment studies over the past two decades. While research on WE has highlighted the status, function, and legitimacy of varieties of English language, it remains unclear how raters respond to the results of the global spread of English. Also unclear is whether their attitudes towards the varieties of English constitute a biasing factor in the scores they award in English speaking tests. As such, this study investigates the relationship between rater attitudes towards Indian English as an example of WE, as measured by the "rater attitude instrument" (RAI), and scores that raters awarded to IELTS speech samples produced by Indian examinees. Methods: A total of 96 teacher raters rated six IELTS speech samples and then completed the RAI online. Correlation analysis, MANOVA, and Tukey contrasts were performed to test the extent to which rater attitudes towards Indian English as an example of WE affect rater scoring decisions on IELTS speech samples. Results: Moderate to strong correlations were observed between the RAI and IELTS speech sample scores. The MANOVA results suggest significantly different ratings, with the positive attitude group consistently awarding higher scores to IELTS speech samples in comparison to the negative attitude group on all of the four analytic rating criteria. Furthermore, the RAI appears to be a significant predictor of IELTS speech sample scores. Conclusion: A link between rater attitude towards Indian English, as an example of WE, and scoring tendency for Indian examinees may exist in a language assessment context. Thus, as raters reoriented their views, broadened their grasp of WE, and as awareness of WE increased in the language testing community in recent decades, the findings here show that testing agencies must add an understanding of potential rater bias towards WE to the current relevant literature.

Critical Literature Review of Contemporary Research on Attitudes Towards World Englishes

“…In contrast with the large number of studies conducted in these countries, however, there are only a few studies conducted in OC countries, particularly in South Asia. Among 43 recent AWEs studies published in international research journals between 2010 and 2020, only four studies were found to be conducted in the South Asian region (Bernaisch, 2012;Bernaisch & Koch, 2016) or foregrounded a South Asian variety (Hsu, 2019;Weekly, 2018).…”

“…Experts in the study of language attitudes like Garret (2010) have pointed out that one of the key assumptions in the ABC model, the causal link between language attitudes, emotions, beliefs and behaviours, has not been satisfactorily established, and thus remains controversial (Garrett, 2010). However, the assumption that a greater awareness of a variety leads to a positive attitude or acceptance of the variety continues to undergird many of the recent AWEs studies conducted in pedagogical contexts (Ahn, 2015;Almagren, 2018;Chan, 2018;Hsu, 2019;Lee, 2019;Sadeghpour & Sharifian, 2017Tsu & Chen, 2014;Wang, 2014). These studies often implicitly and explicitly emphasise the need to develop awareness among teachers and learners, particularly of low-prestige varieties of English, to address negative attitudes towards them, and also to temper the unquestioning demand for IC norms in the classroom.…”