Measuring Socioeconomic Background in PISA: One Size Might not Fit all

Abstract

As part of its flagship educational study -the Programme for International Student Assessment (PISA) -the Organisation for Economic Co-operation and Development (OECD) has undertaken extensive work to create an internationally relevant composite indicator aimed at measuring socioeconomic background. However, the degree to which a single measure of socioeconomic background is reliable and valid for all participating countries is not widely discussed. To fill this gap, the authors examine the home possessions index, which is a key component of PISA's socioeconomic indicator, and highlight a number of issues surrounding this index. In particular, they take a psychometric approach to investigating the reliability and some facets of the validity of the home possessions index in a number of participating PISA countries. Their findings suggest that there are notable concerns with the current index, including highly variable reliability by country, poor modelto-data consistency on a number of subscales, and evidence of poor cultural comparability. They couch their discussion in the context of educational and policy research and propose one possible method for improving these measures for participating countries.

Kosovo’s Low Performance in PISA 2015: An Explanation From a Socioeconomic Perspective

“…Existing literature considered, researchers agree that the variable which explains differences in levels of educational achievement is the socioeconomic status of the family (Rutkowski & Rutkowski, 2013). Beginning with the work of Coleman (1968), who established that socioeconomic status and achievement are linked and that the latter is strongly determined by the first, the relationship has been researched thoroughly as a result of which differences in achievement are mostly attributed to differences in the socioeconomic standing of the family (Evans & Rosenbaum, 2008;Hattie, 2009;Sirin, 2005).…”

Causal inferences with large scale assessment data: using a validity framework

“…For example, missing rates and disagreement between students and parents on identical, policy relevant variables has been shown to be high (see Rutkowski and Rutkowski 2010). Similarly, scale reliabilities can vary widely between countries, from high to unacceptably low (see Rutkowski and Rutkowski 2013).…”

The Contribution of International Large-Scale Assessments to Educational Research: Combining Individual and Institutional Data Sources

“…Critics have argued that the social and cultural pluralism in the construction of education is compressed into a single point in ILSA (e.g., Meyer & Benavot, 2013;Torrance, Lauder, Brown, Dilabough, & Halsey, 2006) and that these scores are subject to methodological limitations (e.g., D. Rutkowski and L. Rutkowski, 2010). These limitations are contentious issues in the literature based on ILSA (Domínguez, Vieira, & Vidal, 2012;Lenkeit, Chan, Hopfenbeck, & Braid, 2015) and we recommend that researchers recognize them when analyzing ISLA data and conduct, if possible, robustness checks.…”