'Naive' native speakers and judgements of oral proficiency in Spanish

Abstract

The notion of the 'native speaker' occupies an important place in any theory of foreign language proficiency. In the case of the ACTFL oral interview, on several occasions the 'native speaker' is invoked as a hypothetical interlocutor for candidates. However, no body of research evidence exists as a foundation for such a use of the concept of 'native speaker'. In an effort to discover whether native speakers do really react to differing levels of proficiency in the way predicted for them in the ACTFL scale, a study was carried out in Barcelona, Spain. A group of linguistically 'naive' native speakers evaluated American students' performances on taped ACTFL oral interviews in Spanish. Apart from the fact that there was a considerable divergence in how they viewed particular candidates' exhibited proficiency, it was striking that the native speakers were consistently more strict in their evaluations than was an ACTFL-trained rater.

Invalidating validity claims for the ACTFL oral rating scale

“…If the tapes used represented samples of speech from students who were well spread out over an ability continuum (even intuitively defined) and also limited in number, such high rank order correlations would not be surprising. The study also fails to provide data concerning the degree to which, once rank ordered, the native and trained raters agreed on the band that described the performance of each of the sample tapes (see Barnwell, 1989, for similar studies on a principled basis).…”

How Do Raters From India Perform in Scoring the Toefl Ibt™ Speaking Section and What Kind of Training Helps?

“…Previous research on the assessment of second or foreign language learners' speaking proficiency by evaluators with different backgrounds has looked at untrained native versus nonnative evaluators (Brodkey, 1972;Caban, 2003;Fayer & Krasinski, 1987;Gorosch, 1973;Kim, 2009;Sheorey, 1985;Smith & Bisazza, 1982;Smith & Rafiqzad, 1979), trained native versus nonnative raters (Brown, 1995), laypeople versus professionals with training in a second or foreign language (Barnwell, 1989;Caban, 2003;Chalhoub-Deville, 1995;Galloway,1980;Gorosch, 1973;Hadden, 1991), male versus female raters (Eckes, 2005), raters with linguistic versus occupational backgrounds (Brown, 1995) However, in explorations of trained rater perceptions in an assessment context, the paramount goal is to reduce potential listener or rater bias and ensure fairness through rater training. In this section, we only review studies that use fairly clear scoring rubrics and raters with training in a second or foreign language or trained raters, which are typically required in an assessment or classroom evaluation context.…”

Exploring Variability in Judging Writing Ability in a Second Language: A Study of Four Experienced Raters of Esl Compositions

“…There are, not surprisingly, differences between professional and lay raters in general, even if, as elsewhere, the findings from research are inconsistent. For example, Barnwell (1989) finds professional raters to be more tolerant of language errors and suggests, not unreasonably, that their exposure to the widest possible range of linguistic ability has enabled them to put learners into a more realistic, thus more forgiving, perspective. Others (e.g., Galloway, 1977;Hadden, 1991) come to the opposite conclusion.…”