Examinee abilities and task difficulty in task-based second language performance assessment

Abstract

T his arlicJc s t•mmarizcs findings from inv~s tigaci on s into the development and usc of n pr<)LOtype English language task-based performance tcs~. D::H~ inc-luded pcrfonmmces by 90 examinees on 13 complex and sk ills ·inlcg ra~ivctasks. a priori estimaliOJlS of examinee proficiency differe nc-es, tt priori esL imaiJons of task dif1iculcy based on cognitive processing demands. pe-rformance ratings accord ing L o ta.sk-SJ)C.Citlc as well as holistic scales a nd criteria, and cxamincesdf-ratings. Findings indicated that the task-based test could infonn intended illferMces about examinees· abi lities to accomplish spcx:ific tasks as wcU as inference..'>aboul examinees' likely abilities with a dom;ain of tasks. A llhough a relationship between task difficulty esLimaL es and examinee pcrformance.s was observed, these estimates were not found ~o provide a lru.stworlhybasis for infe1 ·ringexaminees' likely abiliLies with other tasks. These findings, as weU as srudy limitations, are further discussed in light o f the inLended uses for perfom1ance assessment within kmg\mge education, ~m d rccomrneJldatior\S are made for needed. rcsc.·uch into the interaction bel ween task features. cogniLive processing and l~mguage perronnarlce.

Designing and Scaling Level-Specific Writing Tasks in Alignment With the CEFR: A Test-Centered Approach

“…They found a "lack of consonance" between their results and SLA research findings and came to the conclusion that the "differences between testing and pedagogical contexts are so great as to alter the cognitive focus of the task" (p. 430). This seems further corroborated by Norris, Brown, Hudson, and Bonk (2002), who reported inconsistent results when operationalizing Skehan's model.…”

“…They found a "lack of consonance" between their results and SLA research findings and came to the conclusion that the "differences between testing and pedagogical contexts are so great as to alter the cognitive focus of the task" (p. 430). This seems further corroborated by Norris, Brown, Hudson, and Bonk (2002), who reported inconsistent results when operationalizing Skehan's model.The aforementioned studies seem to indicate that it is difficult to apply task characteristics found in pedagogical contexts to test contexts; nevertheless, these characteristics can help to underpin test constructs and the level descriptors in the CEFR with empirical research. Although a direct alignment of these characteristics to specific CEFR levels is not the aim of the study reported here, it would remain a challenging desideratum for further research.…”

English Language Learning and Technology

“…Every chapter of this book has touched on the importance of our understanding of the nature of the language abilities in the age of technology. In no other area of applied linguistics is the discussion about the best ways of conceptualizing language ability discussed in such a pointed and empiricallybased way as it is in language assessment (e.g., Bachman 2002aBachman , 2002bNorris, Brown, Hudson, & Bonk 2002). As a consequence, it appears that the most promising means of increasing knowledge of these constructs is through theory and research guided by language assessment, at least insofar as language testing researchers heed the warning of the social pragmatists who were worried about tunnel vision: "The way forward is paradoxically to look not ahead, but to look around" (Brown & Duguid 2000: 8).…”

The importance of task variability in the design of learner corpora for SLA research

“…This issue of task variability is also discussed from a language testing perspective where multiple tasks are used to gain a more generalizable picture of learners' language abilities (see Norris et al 2002). The fact that LCR usually uses large samples of learner language goes some way to addressing the issue of construct underrepresentation, yet it still cannot always guarantee a sufficient number of examples of some linguistic feature(s) of current theoretical interest in SLA, as in the case of quantifiers mentioned above.…”