Distributional causal effects: Beyond an “averagarian” view of intervention effects.

Abstract

The usefulness of mean aggregates in the analysis of intervention effectiveness is a matter of considerable debate in the psychological, educational, and social sciences. In addition to studying “average treatment effects,” the evaluation of “distributional treatment effects,” (i.e., effects that go beyond means), has been suggested to obtain a broader picture of how an intervention affects the study outcome. We continue this discussion by considering distributional causal effects. We present formal definitions of causal effects that go beyond means and utilize a distributional regression framework known as generalized additive models for location, scale, and shape (GAMLSS). GAMLSS allows one to characterize an intervention effect in its totality through simultaneously modeling means, variances, skewnesses, kurtoses, as well as ceiling and floor effects of outcome distributions. Based on data from a large-scale randomized controlled trial, we use GAMLSS to evaluate the impact of a teacher classroom management program on student academic performance. Results suggest the teacher classroom management training increased mean academic competence as well as the chance to obtain the maximum score on the academic competence scale. These effects would have been completely overlooked in a traditional evaluation of mean aggregates.

Visual search and childhood vision impairment: A GAMLSS-oriented multiverse analysis approach

“…GAMLSS has been adopted for growth charts (e.g., Borghi et al, 2006), brain charts (e.g., Bethlehem et al, 2022), and reference curves (see the Introduction; see also Durán et al, 2016). More recently, GAMLSS has been featured to model psychological (Campitelli et al, 2017) and educational (Wiedermann et al, 2022) data. As mentioned above, GAMLSS overcomes limitations of other techniques, such as ordinary linear regression and generalized linear regression, by taking care of nonlinear covariates and relating the conditional mean of the response to explanatory variables through distributions other than those of the Exponential family.…”

Beyond the Mean: Can We Improve the Predictive Power of Psychometric Scales?

“…By and large, the calculation of a mean score has been taken for granted, and the consequences of this pervasive practice remain untested. If responses carry meaning beyond their mean, the current practice of using the mean virtually indiscriminately may not be fully leveraging the predictive power of the data (for a like-minded investigation into overall treatment effects, see also Wiedermann et al, 2022).…”