Passageless Comprehension on the Nelson-Denny Reading Test: Well Above Chance for University Students

Abstract

The comprehension section of the Nelson-Denny Reading Test (NDRT) is widely used to assess the reading comprehension skills of adolescents and adults in the United States. In this study, the authors explored the content validity of the NDRT Comprehension Test (Forms G and H) by asking university students (with and without at-risk status for learning disorders) to answer the multiple-choice comprehension questions without reading the passages. Overall accuracy rates were well above chance for both NDRT forms and both groups of students. These results raise serious questions about the validity of the NDRT and its use in the identification of reading disabilities.

Not Read, but Nevertheless Solved? Three Experiments on PIRLS Multiple Choice Reading Comprehension Test Items

“…Pyrczak (1974) reported a substantial amount of passage independence in multiple-choice questions related to the identification of the main idea of singular paragraphs in a sample of graduate students and in a sample of high school students. Coleman et al (2010) reported, as previously mentioned, that examinees answered above chance levels literal comprehension items as well as interpretative comprehension items. To minimize at least the successful use of different test taking strategies while answering multiple-choice items, different authors proposed guidelines for item writers (e.g., Haladyna, 2004).…”

“…Research has provided evidence for the so-called "passage-independence, that is, the likelihood that on some items, an examinee could respond correctly : : : without having read the accompanying passage" (Coleman, Lindstrom, Nelson, Lindstrom, & Gregg, 2010, p. 244). Consequentially, there is a long history of criticism on multiple-choice reading comprehension test items as well (e.g., Coleman et al, 2010;Drum et al, 1981;Katz, Blackburn, & Lautenschlager, 1991;Katz, Lautenschlager, Blackburn, & Harris, 1990;Katz, Marsh, Johnson, & Pohl, 2001;Keenan & Betjemann, 2006;Lifson, Scruggs, & Bennion, 1984;Powers & Leung, 1995;Preston, 1964;Pyrczak, 1972Pyrczak, , 1974Pyrczak, , 1976Rost & Sparfeldt, 2007;Samuels, 1968;Tian, 2006;Tuinman, 1973Tuinman, -1974. Katz et al (1990, Experiment 1), for example, administered 100 multiple-choice reading comprehension test items stemming from the verbal reasoning task of the Scholastic Aptitude Test (SAT) without presenting the corresponding text passages to college students.…”

“…Recently, Coleman et al (2010) administered seven informational text passages (related to humanities, social sciences, and science) and the accompanying 38 multiple-choice questions from the well-known Nelson-Denny Reading Test (Brown, Fishco, & Hanna, 1993) without the corresponding informational text passages to a sample of 253 undergraduate students. Examinees solved about 45% of the items correct (chance level D 20%).…”

“…One factor eventually influencing passage independence might be the differentiation of fictional versus nonfictional text passages. Lifson et al (1984) and Coleman et al (2010) suggested that fictional text passages might be used to reduce passage independence. It seems to be reasonable for nonfictional texts that prior knowledge of the specific content area (the one dealt with in the passage) might result in no or at least less specific gains by reading the passage before answering the questions (and, thereby, substantially reducing the specific advantages of reading the passage in order to answer the questions correctly).…”

Considerações Sobre a Habilidade de Compreensão em Leitura e Formas de sua Avaliação

“…Apesar de o teste possuir bons indicadores psicométricos, tem-se apontado problemas relacionados à validade (p. ex., Coleman, Lindstrom, Nelson, Lindstrom, & Gregg, 2010) e precisão (Murray-Ward, 1998.…”