The Implications of Understanding That PISA Is Simply Another Standardized Achievement Test

Abstract

It occurred to me one day that despite all the excitement, and both the satisfaction and handwringing engaged in by some nations after scores are released, that the Program for International Student Assessment, PISA, is merely another Standardized Achievement Test. Almost all Standardized Achievement Tests (SATs) try to adhere to certain principles of design, have similar correlates, and have similar limits on the interpretations of the results obtained. Neither the popular press nor most politicians ordinarily understand these realities and their implications. Opinion makers are unaware that many of the Standardized Achievement Tests we commonly use do not have the powers attributed to them. It is not far from the truth to call the scores derived from some of these assessments “talismanic” (Haney et al. 1987). That is, for many people, test scores have special powers, particularly of prophesy, a bit like the Kabbalah of the middle ages. Scores from SATs are a part of the metrification associated with the modern world, no doubt aided by a global market place in which business leaders and technologists, instead of humanists and educators, have garnered political power. Contributing to this trend toward metrification has been the ascendance of economics as an influential discipline throughout the world (Lingard et al. 2015). But in my opinion, economists, journalists, and politicians too often seek in metrics powers that are more illusionary than they are real.

Temperament and Academic Achievement in Children: A Meta-Analysis

“…It is also considered to be more sensitive to the individual differences of children [ 9 , 68 ] and to the subjectivity of teachers’ opinions [ 43 ]. Compared to GPA, standardized testing is assumed to be a more objective and a useful measure of students’ achievement [ 43 , 69 ]; however, its ability to adequately capture acquired knowledge and skills is limited [ 70 , 71 ]. Studies have shown that certain individual characteristics of children are more sensitive to a particular assessment method of achievement [ 72 , 73 ].…”

What explains Macau students’ achievement? An integrative perspective using a machine learning approach (¿Cuál es la explicación del rendimiento de los estudiantes macaenses? Una perspectiva integradora mediante la adopción del enfoque del aprendizaje automático)

“…In the extant research, however, there are relatively few studies that have investigated Macau students. The majority of research on PISA and reading, in particular, has been conducted in North America and Western Europe (e.g., Berliner 2020; Manu et al, 2021). This study examined in detail the role of different factors in the Macau context.…”

“…Sin embargo, en la literatura existente existen muy pocos estudios que investigan al estudiantado de Macao. La mayoría de las investigaciones en torno a PISA y la lectura, en particular, se han llevado a cabo en Norteamérica y Europa occidental (e.g., Berliner, 2020; Manu et al, 2021). En nuestro estudio analizamos en detalle el papel de diversos factores en el contexto de Macao.…”

Semi‐automatic coding of open‐ended text responses in large‐scale assessments

“…Some items feature dichotomous coding assignments (correct/incorrect), and some feature three‐level ordinal scaled scoring. The advantages of data from established large‐scale assessments are that the measurement instruments have been tested and optimized using various high‐quality criteria (Berliner, 2020). We also removed empty responses.…”

“…and optimized using various high-quality criteria (Berliner, 2020). We also removed empty responses.…”