Changing times, changing needs: enhancing the utility of international large-scale assessments

Kirsch, Irwin S.; Braun, Henry

doi:10.1186/s40536-020-00088-9

Cited by 10 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although his focus was on the U.S. National Assessment of Educational Progress (NAEP), the issues are germane to international large-scale assessments (ILSA). Over the last 20 years, data generated by ILSA (e.g., TIMSS, PIRLS, PISA) have become more salient in education policy discussions around the globe (Braun & Kirsch, in press; Kirsch & Braun, 2020). 19 In fact, participation in ILSAs can bring many benefits.…”

Section: Narratives and Counternarrativesmentioning

confidence: 99%

Expertise on Offer: Why Isn’t Anyone Buying?

Braun

2023

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

It is a much-lamented fact that research with the potential to inform or influence education policy instead remains policy inert. There are many reasons for this frustrating state of affairs, including a lack of strategic thinking on the part of researchers on how to successfully accomplish outreach—as opposed to communication with peers (in-reach). Another, and a principal focus of this article, is the failure of researchers to appreciate the power of employing compelling narratives to bring their findings to the attention of policymakers and other stakeholders. Accordingly, this article presents some examples of narratives specifically designed for outreach and discusses some of their features. It also considers the challenges in gaining traction with counternarratives once a particular narrative has achieved currency. Researchers should also be mindful of the tenor of the times, with experts now often viewed with skepticism, if not downright hostility. In some quarters, excessive reliance on technocrats is even seen as a threat to democratic governance. The article concludes with some recommendations on how to appropriately enhance the role of research in education policymaking.

show abstract

Section: Narratives and Counternarrativesmentioning

confidence: 99%

Expertise on Offer: Why Isn’t Anyone Buying?

Braun

2023

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Historically quality is associated with industry when an industrially made product had to meet certain quality standards . In education the concept of "quality" is more complicated than just meeting the fixed quality standards, as education quality depends on the needs of an always-changing society and processes that are closely related to this change, therefore measuring the quality of education can be quite challenging (Kirsch & Braun, 2020). Quality of education includes a variety of indicators at various levels (Crissien-Borrero et al, 2019;Sulis et al, 2020).…”

Section: Assessing Quality Of Educationmentioning

confidence: 99%

Quality of Education in Latvian Municipalities and State Cities – Results of International Studies and State Examinations

Geske¹,

Kiselova²,

Pole³

2022

Human, Technologies and Quality of Education, 2022

View full text Add to dashboard Cite

In 2021 a new Law on Administrative Territories and Populated Areas came into force in the Republic of Latvia. To reduce fragmentation, the number of municipalities and State cities was reduced from 119 to 43. There were no changes in 11 local governments, however, other new structures were formed by merging two to eight local governments. On the one hand, these changes are creating new challenges in the education process and institution management, but on the other hand – larger local governments with larger numbers of schools and students opens up new opportunities. One of the opportunity is to make a more accurate assessment of student achievement, which characterizes the performance of a local government’s educational institutions. The aim of this article is to show that despite the great differences between local governments (e. g. population differences from 3 to 614 thousand) it is possible to assess the quality of education in local governments by using data from state examinations and international comparative education studies. Over the last few years Latvia has taken part in IEA (International Association for the Evaluation of Educational Achievement), ICCS (International Civic and Citizenship Education Study), PIRLS (Progress in International Reading Literacy Study), and TIMSS (Trends in International Mathematics and Science Study), as well as in OECD PISA (Programme for International Student Assessment). To obtain the results the data were used from all mentioned studies. This article was supported by European Social Fund project No. 8.3.6.2/17/I/001

show abstract

“…We chose these ELSAs because their IPD have been widely used in secondary data analyses (Drent et al, 2013;Hernández-Torrano & Courtney, 2021;Hopfenbeck et al, 2018;Lenkeit et al, 2015;Reilly et al, 2015, there is extensive coverage of the results of these ELSAs in the media (Hopfenbeck & Görgen, 2017;Johansson, 2016;Steiner-Khamsi et al, 2018), and the results of these ELSAs are intended to inform educational policies (Beaton & Zwick, 1992) and are also likely to affect such policies (Cordero et al, 2018;Kirsch & Braun, 2020;Tobin et al, 2016). The relevance of these ELSAs was, for example, also emphasized by Kirsch and Braun (2020), who conclude that "assessments such as TIMSS, PIRLS, and PISA are arguably the largest and most widely discussed comparative international assessments" (p. 5). Thus, if there are meta-analyses that explicitly rely on data from ELSAs, there is a high likelihood that at least one of these four ELSAs was used in them.…”

Section: Osm1 Literature Search For Meta-analyses Of Educational Large-scale Assessmentsmentioning

confidence: 99%

Meta-Analyzing Individual Participant Data From Studies With Complex Survey Designs: A Tutorial on Using the Two-Stage Approach for Data From Educational Large-Scale Assessments

Brunner¹,

Keller²,

Stallasch³

et al. 2021

Preprint

View full text Add to dashboard Cite

Descriptive analyses of educational phenomena are a vital component of educational research. Such analyses yield reliable results when using representative individual participant data (IPD) from educational large-scale assessments (ELSAs). The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of educational phenomena. This paper offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical complexities of ELSAs (e.g., sampling weights, clustered and missing IPD, dependent effect sizes), first, for conducting descriptive analyses (Stage 1), and second, for integrating the results with meta-analytic and meta-regression models (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from PISA in order to integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students’ SES), and interactions between educational characteristics at the participant level (e.g., the interaction between gender and SES).

show abstract

Changing times, changing needs: enhancing the utility of international large-scale assessments

Cited by 10 publications

References 37 publications

Expertise on Offer: Why Isn’t Anyone Buying?

Expertise on Offer: Why Isn’t Anyone Buying?

Quality of Education in Latvian Municipalities and State Cities – Results of International Studies and State Examinations

Meta-Analyzing Individual Participant Data From Studies With Complex Survey Designs: A Tutorial on Using the Two-Stage Approach for Data From Educational Large-Scale Assessments

Contact Info

Product

Resources

About