On applications of Rasch models in international comparative large-scale assessments: a historical review

Wendt, Heike; Bos, Wilfried; Goy, Martin

doi:10.1080/13803611.2011.634582

Cited by 16 publications

(10 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As a case in point, consider international comparative large-scale assessments like PISA (Programme for International Student Assessment) or PIRLS (Progress in International Reading Literacy Study). These assessments basically rely on testlet-based tests analyzed in a traditional way using Rasch or IRT models (e.g., Monseur, Baye, Lafontaine, & Quittre, 2011; Chang & Wang, 2010; for an overview, see Wendt, Bos, & Goy, 2011).…”

Section: Psychometric Approachesmentioning

confidence: 99%

Examining testlet effects in the TestDaF listening section: A testlet response theory modeling approach

Eckes¹

2013

Language Testing

View full text Add to dashboard Cite

Testlets are subsets of test items that are based on the same stimulus and are administered together. Tests that contain testlets are in widespread use in language testing, but they also share a fundamental problem: Items within a testlet are locally dependent with possibly adverse consequences for test score interpretation and use. Building on testlet response theory (Wainer, Bradlow, & Wang, 2007), the listening section of the Test of German as a Foreign Language (TestDaF) was analyzed to determine whether, and to which extent, testlet effects were present. Three listening passages (i.e., three testlets) with 8, 10, and 7 items, respectively, were analyzed using a two-parameter logistic testlet response model. The data came from two live exams administered in April 2010 (N = 2859) and November 2010 (N = 2214). Results indicated moderate effects for one testlet, and small effects for the other two testlets. As compared to a standard IRT analysis, neglecting these testlet effects led to an overestimation of test reliability and an underestimation of the standard error of ability estimates. Item difficulty and item discrimination estimates remained largely unaffected. Implications for the analysis and evaluation of testlet-based tests are discussed.

show abstract

Section: Psychometric Approachesmentioning

confidence: 99%

Examining testlet effects in the TestDaF listening section: A testlet response theory modeling approach

Eckes¹

2013

Language Testing

View full text Add to dashboard Cite

show abstract

“…Para lograr este fin, se seleccionaron, tradujeron y adaptaron un grupo inicial de ítems que miden rv, rn y re propuestos por Russell y Carter (2015). Luego se procedió a la calibración de los ítems mediante el modelo de Rasch, el cual aporta importantes avances en el desarrollo de evaluaciones a gran escala utilizadas en el ámbito educativo (Wendt, Bos, & Goy, 2011), ya que permite ubicar en una misma escala los parámetros de los ítems y las personas y, en consecuencia, obtener evaluaciones invariantes adaptadas al nivel de competencia de las personas (Prieto & Delgado, 2003).…”

Section: Introductionunclassified

Calibración de un banco de Ítems mediante el modelo de Rasch para medir razonamiento numérico, verbal y espacial

Ghío

Morán

Garrido

et al. 2020

Av. Psicol. Latinoam.

View full text Add to dashboard Cite

En el ámbito educativo, las pruebas de inteligencia son consideradas una de las mejores predictoras del rendimiento académico de los estudiantes. El propósito de este estudio es la adaptación de un grupo de reactivos y la conformación de un banco de ítems (bi) que permita evaluar de manera precisa y objetiva algunas aptitudes cognitivas específicas (razonamiento verbal, numérico y espacial) y generar un indicador general de inteligencia. Para ello, se seleccionaron, tradujeron y administraron 255 preguntas del bi propuesto por los autores Russell y Carter. La muestra estuvo compuesta por 1140 estudiantes pertenecientes a la Universidad Nacional de Córdoba (Argentina), 616 del sexo femenino, 392 del sexo masculino y 132 que no reportaron el sexo, con edades comprendidas entre los 17 y 49 años (M = 20.29; DE = 3.25). Los datos se analizaron mediante el modelo de Rasch. Los resultados expresan que los ítems, en general, poseen adecuadas propiedades psicométricas tanto para las aptitudes específicas, como para los ítems que conforman un indicador general de inteligencia. Se recomienda la utilización del modelo de Rasch para la construcción o adaptación de pruebas y se discuten las implicancias de la utilización de este modelo en las pruebas de aptitudes cognitivas.

show abstract

“…In the past six decades many assessments were undertaken, with many of them under auspices of the IEA. The Organisation for Economic Cooperation and Development (OECD), responsible for the widely-known Programme for International Student Assessment (PISA), also contributed to the development of ILSAs by serving as a forum for international comparative educational research (Wendt, Bos, & Goy, 2011).…”

Section: International Large-scale Assessmentsmentioning

confidence: 99%

“…to "is the intended construct measured the same and with equal precision across participating educational systems?" (Wendt et al, 2011). Making valid comparisons across populations and (sub)populations is fundamental for ILSA outcomes as a starting point for policy making.…”

Section: Validitymentioning

confidence: 99%

“…A booklet rotation system, in which students are administered a sample of items, allows for a larger total set of items to increase domain coverage. IRT also serves the psychometric goal, already stated at the start of ILSAs, namely "to see whether some indications of the intellectual functioning behind responses to short answer tests could be deduced from an examination of the patterning of such responses for many countries" (as stated in Wendt et al, 2011). Wendt et al provide a general overview of IRT methodology and software used in recent ILSAs.…”

Section: Item Response Theorymentioning

confidence: 99%

See 1 more Smart Citation

Improving the modelling of response variation in international large-scale assessments

Punter¹

View full text Add to dashboard Cite

Evolving from early ILSAs such as the First International Mathematics and Science Study (FIMSS; Husén, 1967), TIMSS has the longest tradition and is IEA's most wellknown ILSA, with 49 participating countries in recent cycles (Mullis, Martin, Foy, & Hooper, 2016). Every four years it assesses students in Grade 4 and 8 on their math and science skills. TIMSS is often said to be "curriculum-based", because it uses a curriculum model consisting of the intended, implemented and attained curriculum. These three aspects represent, respectively, what students are expected to learn according to countries' curriculum policies, the actual teaching in classrooms, and, finally, the achievement level of the students and their attitudes regarding the subjects (Mullis & Martin, 2013). To measure these levels, the set of instruments consists of a curriculum, school, teacher, student and, since 2011, home questionnaire in addition to the math and science test. Up until TIMSS-2015 the test was administered using paper-based booklets, though the transition to digital assessment is planned for upcoming cycles (Mullis & Martin, 2017). Similar to TIMSS regarding sampling design is PIRLS. Since 2001, PIRLS assesses the reading literacy of students in Grade 4. The reading test is centred around two reading Nevertheless, to best investigate the validity of a cross-national measurement, the response patterns should also be studied with a measurement model. This can be done to ensure that the precautions in test construction and administration have indeed resulted in items that show the same measurement properties across groups, that is, to ensure measurement invariance. Potential differences in item response behaviour for students of equal ability can complicate inferences regarding proficiency differences between countries or specific student populations. A lack of measurement invariance, characterized by these differences in response behaviour is called differential item functioning (DIF). The issue of valid measurement in secondary analyses of both test and questionnaire data across (sub)populations, is central to this thesis. Attention is directed at the modelling of item responses across (sub)populations, particularly at dealing with potential DIF. The framework of item response theory (IRT; see, for example, Van 1.4 Research Objectives This thesis comprises of several studies that centre around analyses on ILSA data using the framework of IRT for modelling the constructs of interest. Each study is driven by a substantive question from the field of educational research, where the unique properties of ILSA data, such as standardized measurements across countries, and the use of advanced IRT modelling may contribute in an innovative way. In addition, the studies aim to provide new approaches to study validity issues, particularly DIF. Models in this dissertation will be estimated both within a frequentist and Bayesian framework. 1.5 Outline of the Thesis This thesis continues in Chapter 2 with the modelling of computer and information l...

show abstract

On applications of Rasch models in international comparative large-scale assessments: a historical review

Cited by 16 publications

References 47 publications

Examining testlet effects in the TestDaF listening section: A testlet response theory modeling approach

Examining testlet effects in the TestDaF listening section: A testlet response theory modeling approach

Calibración de un banco de Ítems mediante el modelo de Rasch para medir razonamiento numérico, verbal y espacial

Improving the modelling of response variation in international large-scale assessments

Contact Info

Product

Resources

About