Robust statistical methods: A primer for clinical psychology and experimental psychopathology researchers

Abstract

This paper reviews and offers tutorials on robust statistical methods relevant to clinical and experimental psychopathology researchers. We review the assumptions of one of the most commonly applied models in this journal (the general linear model, GLM) and the effects of violating them. We then present evidence that psychological data are more likely than not to violate these assumptions. Next, we overview some methods for correcting for violations of model assumptions. The final part of the paper presents 8 tutorials of robust statistical methods using R that cover a range of variants of the GLM (t-tests, ANOVA, multiple regression, multilevel models, latent growth models). We conclude with recommendations that set the expectations for what methods researchers submitting to the journal should apply and what they should report. Keywords Robust statistical methods, assumptions, bias ROBUST ESTIMATION 3 Robust statistical methods: a primer for clinical psychology and experimental psychopathology researchers OverviewThe general linear model (GLM), which is routinely used in clinical and experimental psychopathology research, was once thought to be robust to violations of its assumptions. However, based on hundreds of journal articles published during the last fifty years, it is well established that this view is incorrect. Moreover, modern methods for dealing with the violations of these assumptions can result in substantial gains in power as well as a deeper, more accurate and more nuanced understanding of data. We begin with an overview of the key assumptions underlying the GLM. We then review various misconceptions about how robust the GLM is to violations of those assumptions and look at the effects that violations can have. We end the first section by looking at the evidence that psychological data, in general, are likely to violate the assumptions of the GLM.In part 2 of the paper we overview a selection of ways to deal with violations of assumptions that fall under the headings of data transformation, adjustments to standard errors, and robust estimation. In the final part, we present 8 tutorials that use datasets relevant to this journal to show how to implement a selection of techniques (robust estimators for model parameters and standard errors) for designs common to this journal (comparing dependent and independent means, predicting continuous outcomes from continuous predictors and longitudinal designs). ROBUST ESTIMATION 4The assumptions of the general linear model Critical assumptionsPsychology researchers (generally) and those with interests in psychopathology (specifically) typically apply variants of the general linear model to their data. In this model, an outcome variable (Y) is predicted from a linear and additive combination of one or more predictor variables (X). For each predictor there is a parameter that is estimated from the data (!) that represents the relationship between the predictor and outcome variable if the effects of other predictors in the model are held constant. There ...

Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change (Preprint)

“…Second, the current findings support a well-established statistical idea posing that the selection of a statistical analysis must match the characteristics of the dataset in order to arrive at valid and accurate statistical measurement, interpretation and conclusions (Flay et al, 2005;Field & Wilcox, 2017 (Castellani et al, 2016;Panagiotakopoulos et al, 2010).…”

“…This use of effect sizes has resulted in both enormous amounts of aggregated evidence about the effects of psychotherapy (Newby et al, 2015), and for this reason, it is understandable clinical researchers would continue to use this standard for measuring and interpreting symptom change. However, should symptom change occur as a proportional function, the measurement and interpretation of treatment related change would substantially improve by matching appropriate statistical analysis to the characteristics of the function of symptom change (Baldwin et al, 2016;Field & Wilcox, 2017;Verkuilen & Smithson, 2012). A possible solution to this dilemma would be to report both the effect size and percentage estimates of change side by side.…”

Aggression toward teachers, interaction with school shooting media, and secondary trauma: Lockdown drills as moderator

“…Residual plots were reviewed to investigate normality, linearity, and heteroscedasticity. Lack of normality was observed for several variables, which is not uncommon for clinical variables (see Field & Wilcox, 2017). Thus, bootstrapping methods were used when calculating descriptive statistics to address problems associated with the lack of normality.…”

Influence of pain anticipation on brain activity and pain perception in Gulf War Veterans with chronic musculoskeletal pain

“…Normally distributed data are presented as means (SD), and non-normal data are presented as 20% trimmed means (20% Winsorized SD; Field & Wilcox, 2017;Mair & Wilcox, 2019;Wilcox, 2017).…”