Studies of the Seriousness of Three Threats to Passage Dependence

Abstract

The three studies presented are concerned with three kinds of multiple-choice reading comprehension questions that may be answered correctly at a higher-than-chance level when they are administered without the accompanying passage. An attempt was made to demonstrate that each of these kinds of high risk item does not necessarily lead to passage dependence invalidity. One study succeeded whereas two failed. The net results of these failures to show that the threats to validity are not necessarily damning was interpreted as evidence of the seriousness of the threats.EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 1980, 40 READING comprehension test questions that can be answered correctly at a higher-than-chance level when administered without the passage(s) upon which they are based seem to be of dubious validity as measures of reading comprehension. However, the ability of examinees to answer items with the passage(s) out (PO) does not demonstrate that examinees, when taking the test with the passage(s) in (PI), actually do answer such items without reading and mentally processing the information on which the items are based. For this reason, the answerability of questions with their PO &dquo;is but a threat to valid measurement and not proof of invalidity&dquo; (Tuinman, 1973(Tuinman, -1974). Yet this threat has been taken by prominent writers (e.g., Pyrczak, 1975; Traxler, 1965, p. 792; Tuinman, 1970) as a very serious indictment of the invalidity of items. Pyrczak (1976, p. 919) listed three kinds of items that examinees may be able to answer with the PO with greater-than-chance success. These are: (1) items that tap general knowledge rather than new information contained in the passage, (2) interrelated questions that clue at University of Manitoba Libraries on June 18, 2015 epm.sagepub.com Downloaded from

A Comparison of Equal Percentile and Partial Credit Equatings for Performance‐Based Assessments Composed of Free‐Response Items

“…Second, clustering of items around a common task is known to lead to some level of response dependency (Ferrara, Huynh, & Baghi, 1992;Yen, 1992). Response dependency has also been noted for multiple-choice items based on a common reading passage (Green & Langhorst, 1986;Hanna & Oaster, 1980;Nicholas & Brookshire, 1987;Scherich & Hanna, 1977).…”