Voices from stakeholders: What makes an academic English test ‘international’?

Gu, Lin; So, Youngsoon

doi:10.1016/j.jeap.2014.10.002

Cited by 28 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The findings were mixed: though WE was favored by the majority on a conceptual level, the inclusion of WE-based linguistic features in tests was less supported for reasons of “maintaining standards, fairness, equality, and test-taker interests” (p. 277). A similar test-takers’ orientation to WE was also found in Gu and Yo’s (2014) study using four stakeholder groups, including test-takers, ESL/EFL teachers, score users, and language testing professionals. Looking at the test-takers’ attitude–test performance relationship, Harding (2008) showed that in general test-takers’ attitudes towards speakers with L2 accents of English on an academic listening test were reasonably positive, and that there was no clear relationship between the attitudes towards speakers and their performance on a listening test.…”

Section: Theoretical Backgroundsupporting

confidence: 77%

Removing bias towards World Englishes: The development of a Rater Attitude Instrument using Indian English as a stimulus

Hsu

2015

Language Testing

View full text Add to dashboard Cite

This study explores the attitudes of raters of English speaking tests towards the global spread of English and the challenges in rating speakers of Indian English in descriptive speaking tasks. The claims put forward by language attitude studies indicate a validity issue in English speaking tests: listeners tend to hold negative attitudes towards speakers of non-standard English, and judge them unfavorably. As there are no adequate measures of listener/rater attitude towards emerging varieties of English in language assessment research, a Rater Attitude Instrument comprising a three-phase self-measure was developed. It comprises 11 semantic differential scale items and 31 Likert scale items representing three attitude dimensions of feeling, cognition, and behavior tendency as claimed by psychologists. Confirmatory factor analysis supported a two-factor structure with acceptable model fit indices. This measure represents a new initiative to examine raters' psychological traits as a source of validity evidence in English speaking tests to strengthen arguments about test-takers' English language proficiency in response to the change of sociolinguistic landscape. The implications for norm selection in English oral tests are also discussed. Theoretical background World EnglishesNew lines of sociolinguistic research have acknowledged the pluricentricity of English such as world Englishes (WE) and English as lingua franca (ELF). The Kachru-led line of WE documents the function, status, linguistic maturity, and legitimacy of the emerging Unlike the constructs being conceptualized in the field of psychology, recent studies on rater attitude towards WE have revealed mixed and inconclusive findings. Kim (2005) examined the language backgrounds of raters, their attitudes toward WE, and how they scored the speech performance of six Korean students on the Test of Spoken English (TSE) picture description task, using holistic and analytic scales. Although their ratings on the holistic scales were fairly similar, their different attitudes towards WE significantly affected their analytic ratings on grammar, rate of speech, and task fulfillment, with those labeled as "positive" giving more lenient ratings.Chalhoub-Deville and Wigglesworth (2005) investigated the rating performance by raters from the inner circle countries, including Australia, Canada, the UK and the US, and found no significant difference in evaluating ESL test-takers' speaking performance.

show abstract

Section: Theoretical Backgroundsupporting

confidence: 77%

Removing bias towards World Englishes: The development of a Rater Attitude Instrument using Indian English as a stimulus

Hsu

2015

Language Testing

View full text Add to dashboard Cite

show abstract

“…There is evidently a need to move away from the large-scale testing solely premised on the monolithic notion of standardized English as used by the idealised user. Although we acknowledge that international testing organisations such as ETS and Pearson recognize the need to incorporate diverse accents and styles of English into current test design to make them more internationally acceptable (see, for example, Gu & So, 2015), we argue that accepting a wider range of language varieties does not go far enough. Focussing on language varieties (e.g.…”

Section: Discussionmentioning

confidence: 93%

English for Academic Purposes: A need for remodelling

Leung

Lewkowicz

Jenkins

2016

Englishes in Practice

View full text Add to dashboard Cite

English for Academic Purposes (EAP) is an established domain of research, teaching, and assessment within additional/second language education. In this article we examine the conceptualisation of English that underpins much of its current thinking and pedagogic practice, and raise questions of validity and claims of 'fit-for-purpose'. In particular we explore issues underpinning EAP assessment and argue that there is a need to reconceptualise the basis of the language model. We propose that given the complex and changing practices in academic communication, there is a good case for broadening the established understanding of Academic English to better reflect target language use. The principles and arguments underlying this discussion are relevant to assessment as well as to EAP more broadly.

show abstract

“…The review of relevant attitude studies on language testing shows a firm conclusion that a listener's attitude and evaluation of a speaker's variety, or competency, cannot be drawn due to different listeners (e.g., rater, student, examinee) used as participants, and a lack of consistent measurement tool to evaluate a listener's attitude towards a speaker's variety. Although the review shows that examinees' attitudes towards WE are generally positive (Hamid 2014;Harding 2008;Gu and So 2014), and their test performance is not associated with their attitudes, raters seem to differ from this preliminary finding (Kim 2005). As the rater is a decisive factor in speaking test scores, the review indicates a pressing need to undertake more empirical studies to look into raters' psychological traits and evaluate the extent to which these traits constitute a potential rater biasing factor.…”

Section: Attitude-behavior Relationship In Language Testingmentioning

confidence: 92%

“…An increasing number of studies place stakeholders' perceptions of WE as a core research concern. While IELTS examinees (Hamid 2014) and various TOEFL stakeholders (Gu and So 2014) indicate support of WE, their attitude may be context-and issue-dependent. The majority of participants in both studies have expressed reservations about the inclusion of WE in the testing situation, particularly in regard to the inclusion of accents and written conventions.…”

Section: Attitude-behavior Relationship In Language Testingmentioning

confidence: 99%

Rater attitude towards emerging varieties of English: a new rater effect?

Hsu

2019

Lang Test Asia

View full text Add to dashboard Cite

Background: A strong interest in researching World Englishes (WE) in relation to language assessment has become an emerging theme in language assessment studies over the past two decades. While research on WE has highlighted the status, function, and legitimacy of varieties of English language, it remains unclear how raters respond to the results of the global spread of English. Also unclear is whether their attitudes towards the varieties of English constitute a biasing factor in the scores they award in English speaking tests. As such, this study investigates the relationship between rater attitudes towards Indian English as an example of WE, as measured by the "rater attitude instrument" (RAI), and scores that raters awarded to IELTS speech samples produced by Indian examinees. Methods: A total of 96 teacher raters rated six IELTS speech samples and then completed the RAI online. Correlation analysis, MANOVA, and Tukey contrasts were performed to test the extent to which rater attitudes towards Indian English as an example of WE affect rater scoring decisions on IELTS speech samples. Results: Moderate to strong correlations were observed between the RAI and IELTS speech sample scores. The MANOVA results suggest significantly different ratings, with the positive attitude group consistently awarding higher scores to IELTS speech samples in comparison to the negative attitude group on all of the four analytic rating criteria. Furthermore, the RAI appears to be a significant predictor of IELTS speech sample scores. Conclusion: A link between rater attitude towards Indian English, as an example of WE, and scoring tendency for Indian examinees may exist in a language assessment context. Thus, as raters reoriented their views, broadened their grasp of WE, and as awareness of WE increased in the language testing community in recent decades, the findings here show that testing agencies must add an understanding of potential rater bias towards WE to the current relevant literature.

show abstract

Voices from stakeholders: What makes an academic English test ‘international’?

Cited by 28 publications

References 19 publications

Removing bias towards World Englishes: The development of a Rater Attitude Instrument using Indian English as a stimulus

Removing bias towards World Englishes: The development of a Rater Attitude Instrument using Indian English as a stimulus

English for Academic Purposes: A need for remodelling

Rater attitude towards emerging varieties of English: a new rater effect?

Contact Info

Product

Resources

About