Robustness of linear mixed‐effects models to violations of distributional assumptions

Abstract

Linear mixed‐effects models are powerful tools for analysing complex datasets with repeated or clustered observations, a common data structure in ecology and evolution. Mixed‐effects models involve complex fitting procedures and make several assumptions, in particular about the distribution of residual and random effects. Violations of these assumptions are common in real datasets, yet it is not always clear how much these violations matter to accurate and unbiased estimation. Here we address the consequences of violations in distributional assumptions and the impact of missing random effect components on model estimates. In particular, we evaluate the effects of skewed, bimodal and heteroscedastic random effect and residual variances, of missing random effect terms and of correlated fixed effect predictors. We focus on bias and prediction error on estimates of fixed and random effects. Model estimates were usually robust to violations of assumptions, with the exception of slight upward biases in estimates of random effect variance if the generating distribution was bimodal but was modelled by Gaussian error distributions. Further, estimates for (random effect) components that violated distributional assumptions became less precise but remained unbiased. However, this particular problem did not affect other parameters of the model. The same pattern was found for strongly correlated fixed effects, which led to imprecise, but unbiased estimates, with uncertainty estimates reflecting imprecision. Unmodelled sources of random effect variance had predictable effects on variance component estimates. The pattern is best viewed as a cascade of hierarchical grouping factors. Variances trickle down the hierarchy such that missing higher‐level random effect variances pool at lower levels and missing lower‐level and crossed random effect variances manifest as residual variance. Overall, our results show remarkable robustness of mixed‐effects models that should allow researchers to use mixed‐effects models even if the distributional assumptions are objectively violated. However, this does not free researchers from careful evaluation of the model. Estimates that are based on data that show clear violations of key assumptions should be treated with caution because individual datasets might give highly imprecise estimates, even if they will be unbiased on average across datasets.

Criteria for acceptable studies of animal personality and behavioural syndromes

“…A suitable compromise may then be to run parametric analyses (e.g., mixed‐effects models) and use bootstrapping or randomization procedures to derive appropriate null distributions for statistical testing. Fortunately, recent research suggests that mixed‐effects models are, in fact, extremely robust to violations of distributional assumptions (Schielzeth et al., 2020).…”

The interacting effect of habitat amount, habitat diversity and fragmentation on insect diversity along elevational gradients

“…In addition, the lmerControl function with the optmizer nloptwrap from the package nloptr was used to improve model performance (Bates, Mächler, Bolker, & Walker, 2015).Model fit was assessed by visual inspection of the residuals. In the models for insect richness and evenness, model residuals revealed slight evidence of non-homogeneity, which was considered as having negligible effects on our model estimates as recently suggested bySchielzeth et al (2020). The absence of data points with high influence was checked by calculating Cook's distances with the function CookD from the predictmeans package.…”

Peer Review #1 of "Perils and pitfalls of mixed-effects regression models in biology (v0.1)"

“…We have provided guidance on a set of typical perils and pitfalls by providing insights from the statistical literature and illustrating key points using simulated case studies and examples. While (G)LMMs can be robust to some of the perils we describe (Schielzeth et al, 2020), there remains little understanding of their combined impacts. Readers can only assess the quality of inference from mixed effects models if their hierarchical structures, levels of true replication, and checks of validity, are described clearly and honestly.…”

“…It is possible to specify alternative hyperparameters and allow random effects to have non-Normal distributions (Zhang et al, 2008;Molenberghs et al, 2010Molenberghs et al, , 2012Fabio, Paula & de Castro, 2012) but this requires non-standard modelling algorithms. In general, both LMMs and GLMMs have been found to be impressively robust against misspecification of the random effects distribution Neuhaus, McCulloch & Boylan, 2013;Schielzeth et al, 2020). The estimates of fixed effect parameters for individual-level variables are particularly robust , while estimation of random effects and variances is more susceptible to misspecification .…”

“…Manuscript to be reviewed (G)LMMs are reassuringly robust to some of the individual pitfalls we have described here (Schielzeth et al, 2020). However, there is very little understanding of the combined impact of multiple violations of the core assumptions: there is plenty of scope for simulation studies of the power and anticonservatism of mixed effects models.…”