The Effects of Surface Structure Variables on Performance in Reading Comprehension Tests

The Prediction of Gre Reading Comprehension Item Difficulty for Expository Prose Passages for Each of Three Item Types: Main Ideas, Inferences and Explicit Statements

“…Instead they seem to measure a reader's world knowledge and his or her ability to reason and think about the contents of a passage ll (p. 162). Royer then cites work by Tuinman (1973Tuinman ( -1974, Drum et al (1981) and Johnston (1984) to bolster this claim. Tuinman's work is similar to the findings of Katz et al (1990) wherein multiple-choice reading items are correctly responded to above chance levels in the absence of the reading passage.…”

“…They reported good predictability using these simple surface variables; on average, they indicated that about 70% of the variance 1 of multiple-choice reading item difficulty was explained. Embretson and Wetzel (1987) also studied the predictability of 75 reading item difficulties using a few of the surface variables studied by Drum et al (1981). But in addition, because of the brevity of their passages, Embretson and Wetzel were able to do a propositional analysis (see Kintsch & van Dijk, 1978) and add variables from this analysis, along with several other measures, as predictor variables.…”

“…Only a few studies appear to have focused on predicting item difficulty using items from standardized ability tests (Drum, Calfee, & Cook, 1981;Embretson & Wetzel, 1987). While not specifically focused on predicting reading item difficulty, many other studies of language processing have isolated a wide variety of variables that influence comprehension difficulty with respect to decision time and recall measures.…”

“…Instead many used dependent measures such as recall of passages or decision time to infer the influence that certain variables have on comprehension difficulty. This review along with our earlier review of the Drum et al (1981) and Embretson &Wetzel (1987) studies will help us to select a final set of variables that we postulate may also index comprehension difficulty within a multiple-choice testing format. Carpenter and Just (1975) found that the occurrence of sentence negation increases comprehension decision time.…”

“…Drum, Calfee, and Cook (1981) predicted item difficulty using various surface structure variables and word frequency measures for the text, and several item variables that also depended on surface structure characteristics (e.g., number of words in the stem and options, number of words with more than one syllable). They reported good predictability using these simple surface variables; on average, they indicated that about 70% of the variance 1 of multiple-choice reading item difficulty was explained.…”