A Contextualized Approach to Describing Oral Language Proficiency

Abstract

Although both raters and elicitation tasks are principal factors influencing the study of learners' second language L2) oral proficiency, the effect of each has always been investigated separately; consequently, any possible relationship between them remains unexplored. In investigating the L2 oral proficiency construct, the present study incorporated a variety of tasks and diverse rater groups. The tasks encompassed an interview, a narration, and a read‐aloud. The rater groups, all NSs of Arabic, included 15 teachers in the US, 3 nonteaching raters residing in the US, and 36 nonteaching raters living in Lebanon. Using multidimensional scaling analyses, I derived dimensions underlying raters' holistic ratings of 6 learners' L2 oral proficiency on each of the three tasks. In addition, I specified the salience of the derived dimensions for each of the three rater groups. The results show that the nature of the L2 oral construct is not constant. Different weighted dimensions emerged when investigating the various tasks and rater groups. I concluded that proficiency researchers should not employ generic dimensions; dimensions should be empirically derived according to the specific elicitation task and audience.

Rater judgment and English language speaking proficiency

“…Further analyses of the present data are warranted. In similar investigations of holistic ratings, Chalhoub-Deville (1995a, 1995b argues that significance tests that look at the means fall short in terms of informing us about the variables that are driving the holistic ratings. In other words, such analyses may tell us that some method or task is more or less difficult for learners or that some rater group is more or less lenient in judging learners' performance.…”

Computer-delivered or face-to-face: effects of delivery mode on the testing of second language speaking

“…It may also be possible that the participants responded to the tasks differently between the modes; however, the raters may have adjusted the possible variability in performance as reported in Brown (2005) and McNamara and Lumley (1997). Some raters in the current study may have awarded a higher rating to the participants, assuming that the interviewer was not helpful in eliciting speech samples.…”

Predicting ESL learners’ oral proficiency by measuring the collocations in their spontaneous speech

“…On the other hand, it is rather presumptuous of Versant tests to generalize about these 'core spoken language skills' given our limited knowledge of the oral construct. As a matter of fact, there is considerable evidence against the invariance of the oral construct in the real world (see e.g., Chalhoub-Deville, 1995;Schmidgall, 2013). For example, highly accurate and intelligible pronunciation is not seen in fast-paced auctioneers' language (Kuiper, 1996); speech accentedness may have a larger effect on comprehensibility among native-speaking listeners than non-native-speaking listeners (Jun & Li, 2010).…”