Language for Specific Purposes assessment criteria: where do they come from?

Abstract

Typically in assessment of Language for Specific Purposes (LSP), test content and methods are derived from an analysis of the target language use (TLU) situation. However, the criteria by which performances are judged are seldom derived from the same source. In this article, I argue that LSP assessment criteria should be derived from an analysis of the TLU situation, using the concept of indigenous assessment criteria (Jacoby, 1998). These criteria are defined as those used by subject specialists in assessing communicative performances of both novices and colleagues in academic, professional and vocational fields. Performance assessment practices are part of any professional culture, from formal, gatekeeping examination procedures, to informal, ongoing evaluation built into everyday interaction. I suggest a procedure for deriving assessment criteria from an analysis of the TLU situation and explore problems associated with doing so, recommending a ‘weak’ indigenous assessment hypothesis to assist in the development of LSP test assessment criteria and guide interpretations of test performance.

The English Immersion Program: Measuring the Communication Outcomes

“…Assessment practices that reflect communicative approaches to language training in Language for Specific Purposes (LSP) contexts is also well covered (Lumley and Brown 1996;McNamara 1997;Douglas 2000;Elder 2001), and well-documented accounts of test development for different groups of occupations and professionals abound. McNamara (1990McNamara ( ,1996Elder (2001) and McDowell (1995) have both developed standardised and performance-based tests for a range of teachers and health professionals in Australia and Douglas (2000) looks at specific language use situations to develop test content and test methods for highly specific LSP, such as English for pilots and air traffic control.…”

“…McNamara (1990McNamara ( ,1996Elder (2001) and McDowell (1995) have both developed standardised and performance-based tests for a range of teachers and health professionals in Australia and Douglas (2000) looks at specific language use situations to develop test content and test methods for highly specific LSP, such as English for pilots and air traffic control. Such frameworks, however, have yet to be fully understood in an industry context and incorporated into business language assessment practices onsite (Lockwood 2002;Lockwood 2008).…”

Language for Specific Purpose (LSP) performance assessment in Asian call centres: strong and weak definitions

“…It would appear that spoken language performances in specific contexts, for example in patient interaction, air traffic controller/pilot exchanges, academic conference presentations; and in this case, call centre exchanges, require a good understanding of that context, a good understanding and analysis of the target texts in that context, and a reliance on subject matter expert (SME) input as to what constitutes 'task fulfillment' to achieve validity in that context. Using the concept of 'indigenous criteria' , meaning what occupational/professional stakeholders value in communicative exchange at work, (see Jacoby, 1998;Jacoby and McNamara, 1999;Douglas, 2001Douglas, , 2005, language testers have looked at different ways and sources from which these criteria may be derived. McNamara (1996) and Jacoby and McNamara (1999), first became interested in how SMEs and language assessors (LAs) differed in their evaluations of successful spoken performance:…”

“…Douglas (2000Douglas ( , 2001 and Hamp-Lyons and Lumley (2001) later argued that the successful sourcing of these professionally relevant criteria to be one of the most urgent needs for research and practice in LSP assessment. McNamara (1996) proposed 'the use of the terms strong and weak second language performance test to reflect the extent to which the assessment criteria reflect real-world contexts, or whether they are oriented mainly to linguistic aspects of the communication respectively' (p. 217).…”

“…Building on existing labellings of 'weak' and 'strong' LSP assessment (see for example McNamara, 1996;Douglas, 2001), this study proposes clear definitions for each of these where BUPLAS is used for recruitment and quality assurance in the Asian call centre context. I argue that the essential differences between the 'weak' and 'strong' versions of BUPLAS are characterized in their specific purposes and impact value; in their locations; in their raters; in their tasks, and ultimately, in the ways in which the common BUPLAS criteria are ranked, weighted and elaborated.…”