An application of the Rasch model to reading comprehension measurement

Abstract

An effective reading comprehension measurement demands robust psychometric tools that allow teachers and researchers to evaluate the educational practices and track changes in students' performance. In this study, we illustrate how Rasch model can be used to attend such demands and improve reading comprehension measurement. We discuss the construction of two reading comprehension tests: TRC-n, with narrative texts, and TRC-e, with expository texts. Three vertically scaled forms were generated for each test (TRC-n-2, TRC-n-3, TRC-n-4; TRC-e-2, TRC-e-3 and TRC-e-4), each meant to assess Portuguese students in second, third and fourth grade of elementary school. The tests were constructed according to a nonequivalent groups with anchor test design and data were analyzed using the Rasch model. The results provided evidence for good psychometric qualities for each test form, including unidimensionality and local independence and adequate reliability. A critical view of this study and future researches are discussed.

Assessing reading comprehension with narrative and expository texts: Dimensionality and relationship with fluency, vocabulary and memory

“…In this article, we present validity evidence for two tests of reading comprehension for Portuguese students in elementary school: the TRC‐n – Test of Reading Comprehension of Narrative Texts and the TRC‐e – Test of Reading Comprehension of Expository Texts. Each test is composed of three vertically scaled forms (TRC‐n‐2, TRC‐n‐3 and TRC‐n‐4; TRC‐e‐2, TRC‐e‐3 and TRC‐e‐4), generated through Rasch model analyses (Santos, Cadime, Viana et al ., ). These test forms were constructed to assess the same construct, and were designed to be as similar as possible in content but were allowed to vary in terms of difficulty, since these tests measure a construct (reading comprehension) that changes with learning and is expected to increase with academic grade levels (Kolen & Brennan, ).…”

“…These test forms were constructed to assess the same construct, and were designed to be as similar as possible in content but were allowed to vary in terms of difficulty, since these tests measure a construct (reading comprehension) that changes with learning and is expected to increase with academic grade levels (Kolen & Brennan, ). Therefore, using a vertical scaling procedure, the scores of each test form were adjusted to a common scale, enabling the comparison of performance in reading comprehension measured with the different test forms (Santos et al ., ). The first study in this article is focused on validity evidence based on the internal structure of each test form and the second one is dedicated to the validity evidence based on relations with other variables, namely reading fluency, vocabulary and working memory.…”

“…This aspect can have a major impact on the information available to educators and psychologists about students' abilities to comprehend a different range of texts with which students are confronted in schools (Morsy, Kieffer & Snow, ). Besides, the construction of vertically scaled test forms allows growth in reading comprehension to be measured from second to fourth grade (Santos et al ., ).…”

Developing and Validating Tests of Reading and Listening Comprehension for Fifth and Sixth Grade Students in Portugal

“…The assessment of these skills allows us to identify at-risk readers, to support the development of intervention and teaching programs, and to monitor the students' progress in these areas over time (Santos et al, 2016b). For this purpose, the use of standardized measures with robust psychometric qualities is essential (Salvia et al, 2010).…”

Applying Rasch Model to Measure Students` Reading Comprehension

“…Aryadoust& Zhang (2016) conducted a study using the Rasch model to determine students' reading skill which are divided into two class groups. While Santos et al, (2016) only presented an analysis of test quality based on psychometric characteristics on reading comprehension tests.…”