The notion of subskills in reading comprehension tests: an EAP example

Abstract

It has been suggested that reading ability can be divided into various subskills, and this notion is common in ESL teaching and testing. It has, however, also been argued (Alderson and Lukmani, 1989) that teachers are unable to reach agreement about the reading subskills which may be tested by particular reading test items. This study begins by examining the place of subskills in ESL syllabus and test design, with particular attention to the enduring influence of the work of Munby (1978). The issue of teachers' perceptions of subskills and their difficulty, as represented in reading comprehension tests, is discussed. A framework is put forward for negotiating agreement between teachers about subskills tested by reading comprehension test items. Using this framework, very substantial agreement between a group of five experienced teachers of EAP is shown to be achieved in matching subskills to individual test items in the reading section of a test of EAP, as well as in judging the difficulty of these subskills. After brief discussion of the use of Rasch IRT in analysis of reading comprehension test items, the teachers' consensus regarding subskill difficulty level is compared to the Rasch analysis of item difficulty, and the significant correlation found gives some empirical validation to the teachers' perceptions. Implications of the findings for analysis of test content, and for teaching, are considered.

The Relative Difficulty and Significance of Reading Skills

“…The results indicated a hierarchy according to which the third subskill was more difficult than the first and the second ones. Likewise, Lumley (1993) arrived at a very considerable agreement among judges on the difficulty level of the subskills in his study as well as a high correlation between statistical evidence on the difficulty of the items and the judges' consensus on their difficulty level.…”

“…Likewise, Lumley (1993) refers to Carroll (1980) who identified language skills as part of test content specification and holds that Munby's taxonomy of language skills has been one of the most productive sources of information for the purpose of test construction. Carroll (1980) considers eleven subskills of the Munby's list as appropriate for testing.…”

“…Carroll (1980) considers eleven subskills of the Munby's list as appropriate for testing. Weir (1988) examining the development of TEEP tried to include discrete assessment of the reading subskills in addition to integrated performance tasks, as noted by Lumley (1993). He offers a list of reading comprehension subskills relevant to EAP contexts.…”

“…Alderson (2000) cites quantitative studies supporting multi-divisible view such as Carroll (1993), Drum et al (1981), Pollitt et al (1985), and Davey (1988). Qualitative studies against a unitary view of reading include Anderson et al (1991), Bachman et al (1988), Brutten et al (1991), Lumley (1993), Weakly (1993) and Weir et al (1990), as cited by Weir and Porter (1994 ). In support of multi-divisible view of reading, Lumley (1993) argues that it may not be possible to say that reading subskills exist objectively as separate, concrete elements.…”

“…Qualitative studies against a unitary view of reading include Anderson et al (1991), Bachman et al (1988), Brutten et al (1991), Lumley (1993), Weakly (1993) and Weir et al (1990), as cited by Weir and Porter (1994 ). In support of multi-divisible view of reading, Lumley (1993) argues that it may not be possible to say that reading subskills exist objectively as separate, concrete elements. However, the fact that the literature is full of reference to them and that they are often considered in syllabus design and test construction suggests that they are inherently appealing as a working construct to many teachers and testers.…”