The Use of Situation Tests as Measures of Communicative Ability

Abstract

In recent years, universities and secondary schools have increasingly used the ACTFL/ETS Oral Proficiency Interview (OPI) to measure the ability of learners to communicate in a foreign language. This article discusses the OPI in relation to current models of communicative skills and argues that the OPI fails to measure important aspects of communicative ability. Two Situation Tests, one written and one oral, are proposed as alternative measures of communicative ability and are described in detail. The two tests as well as the OPI were administered to American university students who had spent a year abroad studying French. This article reports on the changes in the communicative skills of the students during the year after their return to the United States. Statistical comparisons between the OPI and the Situation Tests are presented showing that the OPI is primarily a measure of grammatical competence. The article concludes with the claim that Situation Tests can provide a more complete assessment of communicative ability than the OPI.

Revising the revised format of the ACTFL Oral Proficiency Interview

“…However, Messick (1994: 14) states that 'the portrayal of performance assessments as authentic and direct has all the earmarks of a validity claim but with little or no evidential grounding'. Accordingly, many researchers have argued against the validity of the ACTFL guidelines due to the lack of theoretical and empirical support (e.g., Lantolf and Frawley, 1985;Savignon, 1985;Bachman and Savignon, 1986;Kramsch, 1986;Raffaldini, 1988;Valdman, 1988;Bachman, 1990;Shohamy, 1990). For instance, Shohamy (1990: 386) points out that tests such as the ACTFL-OPI are limited in scope in terms of assessment of developmental stages as well as communicative interaction:…”

“…Along the same lines, Kormos (1999: 165) claims that a more extended use of guided role plays would address the problems of the non-symmetrical nature of interaction in nonguided interviews (see also Raffaldini, 1988). 5…”

“…A possible solution for this dilemma is to do an assessment of sub-components on a principled basis. For instance, it is possible to have a series of tasks (e.g., a series of role plays) that assess language performance in different combinations of sociolinguistic conditions (see Raffaldini, 1988;Shohamy, 1988;Kormos, 1999). In general, the inadequate or inconsistent rating of speech samples is predicated on two main factors: 1) the lack of identification of the criteria used to rate performance within each module of communicative language ability; and 2) the relative weight attributed to the different components of communicative ability (see Section 5 below).…”

Language Testing: Matching Assessment Procedures with Language Knowledge

“…Shohamy, Reves, and Bejerano (1986) introduced an oral test in which test takers were required to use four interactions -an oral interview, a role play, a reporting task, and a group discussion. Raffaldini (1988) describes an oral test which included a role play and a number of situational tasks. Tests such as the TEEP (Test for English for Educational Purposes) and the Royal Society of Arts test are also examples of instruments which attempted to be communicative and authentic.…”

Interlocutor support in oral proficiency interviews: the case of CASE

“…The assessment of L2 speaking proficiency, particularly as measured by the Foreign Service Institute-Interagency Language Roundtable (FSI/ILR) interview (Lowe, 1982), the ACTFL/ETS Oral Proficiency Interview (OPI) (ACTFL, 1986) and their variants, has been a topic of considerable interest to the language testing community over the last 15-20 years, as is evidenced by the extensive body of research on such issues as content validity (Lowe, 1981), construct validity (Bachman and Palmer, 1981;1982;Dandonoli and Henning, 1990;Henning, 1992;Reed, 1992), reliability and rating procedures (Adams, 1980;Shohamy, 1983;Magnan, 1987), comparisons with other oral testing methods (Clark, 1979;Clark and Hooshmand, 1992;Stansfield and Kenyon, 1992), aspects of the communicative competence construct (Byrnes, 1987;Henning and Cascallar, 1992), and other aspects of oral testing (Clark, 1978; 1980; Madsen and Jones, 1981;Clark and Lett, 1988;Raffaldini, 1988;Shohamy, 1988;Pavlou, 1994). A number of researchers have also raised objections to the OPI and its variants (Bachman and Savignon, 1986;Kramsch, 1986;Bachman, 1988;Lantolf and Frawley, 1985;.…”