Appropriate modelling of school compositional effects: a response to Malatinszky and Armor, and Marks

The harms of inequity are rarely overestimated: Comment on Marks & O’Connell 2021

“…It is only in a study by Marks himself in collaboration with David J. Armor and Aron Malatinszky that covariates are interpreted without taking into account the presence of an intermediate outcome, as when they conclude that the "analyses presented in this article do not support the widely held view that school SES and school racial composition have strong effects on student achievement" (Armor et al, 2018). We were apparently not the only ones to notice this -M. Sciffer et al (2021) reply to Marks and his coauthors that "fixed-effects models exclude most of the variation in socioeconomic composition, are incapable of measuring the commonly defined conceptualisation of school composition and can mischaracterise the relative importance of socioeconomic composition" (2021, p. 627; also see M. G. Sciffer et al, 2020).…”

“…• Much of the ostensible evidence against the relevance of social and economic background misinterprets research that estimates gain scores or accounts for prior ability, where the effects of student background on educational outcomes are much smaller because they are residual effects. (M. Sciffer et al, 2021;M. G. Sciffer et al, 2020) • A lot of time is spent arguing that social and economic background is poorly measured but it is not recognized that this leads to systematically underestimated effects and thus argues against their thesis rather than in favor of it.…”

The harms of inequity are rarely overestimated: Comment on Marks and O'Connell (2021)

“…• Misinterpretation of direct effects in models with intermediate outcomes. Much of the ostensible evidence against the relevance of social and economic background consists of research that estimates gain scores or accounts for prior ability, and in such studies the effects of student background on educational outcomes are much smaller because they are residual effects (Sciffer et al, 2021(Sciffer et al, , 2020. • Lack of serious engagement with the research that is cited, with errors that range from references to the wrong work, wrong author or wrong year, work that is appropriated and quoted out of context because only the abstract or conclusion were considered, opportunistic use of research or methods that were criticised mere paragraphs before, and selectiveness in what was cited in a way not indicative of an even-handed review.…”

“…It is only in a study by Marks himself in collaboration with David J. Armor and Aron Malatinszky that covariates are interpreted without accounting for the accumulated disadvantage the fixed‐effect design leaves out, as when they conclude that the ‘analyses presented in this article do not support the widely held view that school SES and school racial composition have strong effects on student achievement’ (Armor et al., 2018). This was first noted by Michael Sciffer and colleagues, who replied to Marks and his coauthors that ‘fixed‐effects models exclude most of the variation in socioeconomic composition, are incapable of measuring the commonly defined conceptualisation of school composition and can mischaracterise the relative importance of socioeconomic composition’ (Sciffer et al., 2021, p. 627; also see Sciffer et al., 2020).…”

“…Furthermore, the authors argue that social and economic background is poorly measured but it is not recognised that this leads to downwardly biased environmental effects (Blakely et al., 2004; Steiner et al., 2011; Westfall & Yarkoni, 2016) that, once disattenuated, are much too large for genetic confounding to be a predominant force even when present. Misinterpretation of direct effects in models with intermediate outcomes. Much of the ostensible evidence against the relevance of social and economic background consists of research that estimates gain scores or accounts for prior ability, and in such studies the effects of student background on educational outcomes are much smaller because they are residual effects (Sciffer et al., 2021, 2020). Lack of serious engagement with the research that is cited, with errors that range from references to the wrong work, wrong author or wrong year, work that is appropriated and quoted out of context because only the abstract or conclusion were considered, opportunistic use of research or methods that were criticised mere paragraphs before, and selectiveness in what was cited in a way not indicative of an even‐handed review. Over‐reliance on the amount of variance explained by heritable traits and the environment, a metric that is highly sensitive to who was measured, when and how, and a poor proxy for the likely responsiveness of educational outcomes to interventions that aim for greater equity (Branigan et al., 2013; Goldberger, 1979; Kempthorne, 1978; Lewontin, 1974; Manski, 2011).…”