Modern international large-scale assessment in education: an integrative review and mapping of the literature

Abstract

Research in international large-scale assessment (ILSA) has become an increasingly popular field of study in education. Consequently, interest and debate in the field by practitioners, researchers, policymakers, and the public has grown over the past decades. This study adopts a descriptive bibliometric approach to map modern research on ILSA in education and provide an up-to-date picture of the recent developments and structure of the field. The analysis of 2,233 journal articles indexed in the Web of Sciences database revealed that ILSA research in education is an emerging field in a stage of exponential growth that has become increasingly international with recent substantive contributions from China, Spain, and Turkey. Research in the field is currently produced by a tupid network of scholars with diverse geographical backgrounds that engage frequently in national and international research collaborations. Also, the field is relatively interdisciplinary and has developed grounded on nine differentiated historical paths. The PISA program has received the greatest attention in the field, and a wide variety of topics have been addressed in the literature in the last decades, including equity and quality education, globalization and education policy, measurement and statistics, student motivation and self-concept, and interpersonal relationships. The paper concludes by pointing to the potential of future ILSA research to make use of new more relevant instrumentation, data linkages, and trans-regional collaborations.

¿Cómo sé que mi secuencia didáctica es de calidad? Propuesta de un marco de evaluación desde la perspectiva de Investigación Basada en Diseño

“…Teniendo en cuenta todo lo anterior, vemos que la propuesta IBD difiere de ideas comunes en evaluación educativa a gran escala (Hernández-Torrano y Courtney, 2021), al no haber exigencia de transferibilidad ni expectativas de generalización estadística de los resultados. A pesar de la importancia que tienen estos tipos de evaluación, estamos de acuerdo con diversos autores que en las investigaciones IBD adquiere una especial relevancia las evaluaciones de "grano fino" ligadas a la calidad interna de las SEA/DI en un contexto concreto de planificación, implementación y obtención de resultados (Guisasola et al, 2021;Plomp, 2013;Psillos y Kariotoglou, 2016).…”

Pakistan’s First ever Participation in International Large-Scale Assessment (TIMSS): Critique and Implications

“…ILSA entails "Studies in which both achievement of certain age/grade in one or more subjects is compared across education systems and effects of contextual factors at system, school, classroom and students' level on achievement" (Bos, 2002, p.2). The emergence of ILSA can be traced to the late 1950s when a group of researchers at UNESCO discussed conducting assessment of students' achievement in several countries (Hernández-Torrano & Courtney, 2021). This discussion resulted in the establishment of the International Association for the Evaluation of Education Ahmed, Rodrigues & Bhutta Achievement (IEA), which conducted the First International Mathematics Study (FMIS) in 1964 to compare the performance of students among 10 participating countries (Rutkowski et al, 2014).…”

“…Apparent justifications for participating in any ILSA include, but are not limited to, evidence-based policy formulation, competing with the globalised world, bringing accountability through standardised results, improving curriculum, teaching in the light of ILSA results, and capacity building of local staff working on assessment (Addey & Sellar, 2018). The factors that motivate countries to participate in ILSA can be broadly explained in a four-dimensional framework which includes: i) the rationale choice model; ii) the policy diffusion model, iii) the macro-dissatisfaction perspective; and, iv) the financial aid model (Hernández-Torrano & Courtney, 2021 ;Kijima, 2010).…”

“…Apart from the above, there are several other benefits of ILSA for participating countries as well as researchers around the world, which includes: curriculum reforms; utilizing strategies; enhancing access and equity; providing access to large-scale representative data at regional and international level; international comparison; and indicators to monitor and evaluate educational processes (Tobin et al, 2016;UNESCO, 2018). Furthermore, researchers conducting secondary analysis on the publicly available ILSA data have made a remarkable addition to publications internationally (Hernández-Torrano & Courtney, 2021).…”

Meta-Analyzing Individual Participant Data From Studies With Complex Survey Designs: A Tutorial on Using the Two-Stage Approach for Data From Educational Large-Scale Assessments

“…In these searches, we looked for documents that reported the application of methods to data from four major ELSAs: NAEP, PISA, TIMSS, and PIRLS. We chose these ELSAs because their IPD have been widely used in secondary data analyses (Drent et al, 2013;Hernández-Torrano & Courtney, 2021;Hopfenbeck et al, 2018;Lenkeit et al, 2015;Reilly et al, 2015, there is extensive coverage of the results of these ELSAs in the media (Hopfenbeck & Görgen, 2017;Johansson, 2016;Steiner-Khamsi et al, 2018), and the results of these ELSAs are intended to inform educational policies (Beaton & Zwick, 1992) and are also likely to affect such policies (Cordero et al, 2018;Kirsch & Braun, 2020;Tobin et al, 2016). The relevance of these ELSAs was, for example, also emphasized by Kirsch and Braun (2020), who conclude that "assessments such as TIMSS, PIRLS, and PISA are arguably the largest and most widely discussed comparative international assessments" (p. 5).…”