Developing and Validating Tests of Reading and Listening Comprehension for Fifth and Sixth Grade Students in Portugal

Abstract

An efficient assessment of reading and linguistic abilities in school children requires reliable and valid measures. Moreover, measures which include specific test forms for different academic grade levels, that are vertically equated, allow the direct comparison of results across multiple time points and avoid floor and ceiling effects. Two studies were conducted to achieve these goals. The purpose of the first study was to develop tests of reading and listening comprehension in European Portuguese, with vertically scaled test forms for students in the fifth and sixth grades, using Rasch model analyses. The purpose of the second study was to collect evidence for the validity of these tests based on the relationships of test scores with other variables. The samples included 454 and 179 students for the first and second study, respectively. The data from both studies provided evidence for good psychometric characteristics for the test forms: unidimensionality and local independence, as well as adequate reliability and evidence of validity. The developed test forms are an important contribution in the Portuguese educational context as they allow for the assessment of students’ performance in these skills across multiple time points and can be used both in research and practice.

Numeracy across grades – vertically scaling the Norwegian national numeracy tests

“…Reading, on the other hand, is less dependent on grade-specific knowledge, and reading has also shown as a skill measurable across multiple grades in both English and other languages (e.g. Australian Curriculum, Assessment and Reporting Authority, 2014;Beaton & Johnson, 1992;Diakidoy, Stylianou, Karefillidou, & Papageorgiou, 2005;Rodrigues et al, 2020). Within the Norwegian context, it is natural to assume the other national tests measuring reading in Norwegian and reading in English could be vertically scaled post-hoc.…”

Assessing oral reading fluency within and across grade levels: Development of equated test forms

“…It is composed of five vertically scaled forms that assesses reading comprehension of narrative texts in students from second to sixth grades (Rodrigues et al, 2020;Santos et al, 2016). Students were instructed to read silently a set of texts and answer a set of multiple-choice questions (three options) about them.…”

Reading Comprehension Tests for Children: Test Equating and Specific Age-Interval Reports

“…We performed a nonsystematic review in PubMed and Science Direct (using the terms, "reading comprehension" and "equating" or "concurrent calibration") and we found only 32 studies. From these, only nine were related to equating reading comprehension tasks with real data (Ree et al, 2003;Betts et al, 2009;von Davier, 2013;Liao et al, 2014;Dimitrov, 2016;Sandefur, 2018;Seo et al, 2018;Wagner et al, 2018;Rodrigues et al, 2020). In general, these studies present a wide diversity of approaches for reporting equating.…”

“…A great part of the research in the educational area used classic or observed scores approaches to equating (Ree et al, 2003;Betts et al, 2009;von Davier, 2013;Wagner et al, 2018). Most part of the studies that used IRT approach used Rasch models, which do not provide information about the discrimination of the items (Liao et al, 2014;Sandefur, 2018;Seo et al, 2018;Rodrigues et al, 2020). The exception is the study of Dimitrov (2016) that used 3PLM.…”

“…The great majority of the revised studies are concerned with multiple-choice tests, with the exception being the studies by Betts et al (2009) that equated a measure of fluency in reading and of Wagner et al (2018) that used a mix of open-constructed and multiple-choice questions from Program for International Student Assessment (PISA). It is also remarkable that the studies generally focus on adult or youth population, with only two studies (Betts et al, 2009;Rodrigues et al, 2020) focusing on younger children.…”