Fixed or random? On the reliability of mixed‐effects models for a small number of levels in grouping variables

Abstract

Biological data are often intrinsically hierarchical (e.g., species from different genera, plants within different mountain regions), which made mixed‐effects models a common analysis tool in ecology and evolution because they can account for the non‐independence. Many questions around their practical applications are solved but one is still debated: Should we treat a grouping variable with a low number of levels as a random or fixed effect? In such situations, the variance estimate of the random effect can be imprecise, but it is unknown if this affects statistical power and type I error rates of the fixed effects of interest. Here, we analyzed the consequences of treating a grouping variable with 2–8 levels as fixed or random effect in correctly specified and alternative models (under‐ or overparametrized models). We calculated type I error rates and statistical power for all‐model specifications and quantified the influences of study design on these quantities. We found no influence of model choice on type I error rate and power on the population‐level effect (slope) for random intercept‐only models. However, with varying intercepts and slopes in the data‐generating process, using a random slope and intercept model, and switching to a fixed‐effects model, in case of a singular fit, avoids overconfidence in the results. Additionally, the number and difference between levels strongly influences power and type I error. We conclude that inferring the correct random‐effect structure is of great importance to obtain correct type I error rates. We encourage to start with a mixed‐effects model independent of the number of levels in the grouping variable and switch to a fixed‐effects model only in case of a singular fit. With these recommendations, we allow for more informative choices about study design and data analysis and make ecological inference with mixed‐effects models more robust for small number of levels.

Mixed-Effects Regression Weights for Advice Taking and Related Phenomena of Information Sampling and Utilization

“…Observations with c i = 1 were excluded as no advice was provided. Technically, only Experiment 2 of Mayer and Heck (2022) satisfies the practical recommendations about a minimum number of five factor levels for precise estimation of random effects variances (Bolker, 2015; see also Oberpriller et al, 2022).…”

Concurrent ordination: simultaneous unconstrained and constrained latent variable modeling

“…For the second example the predictors were unrelated to one of the latent variables. In such situations the concurrent ordination can be simplified by omitting the LV-level error or predictors for some dimensions, and doing so is likely to improve convergence of the models (Oberpriller et al 2022).…”

“…Mixed-effects models with scale parameters on the boundary of the feasible parameter space usually suffer from numerical issues (Oberpriller et al 2022), and in a realistic workflow the model should have been refitted…”

Numerical Response of Owls to Dampening of the Population Cycles of the Small Mammals in Latvia

“…In species with repeated samples from the same nest sites, we also created linear mixed effects models (LMMs) with nest ID as a random intercept. In the case of singularity in the random part, we used the result of LM [71][72], as there were too few replicates per nest to contribute to the model fit. Model coefficients in fixed part did not differ in these models.…”

“…Additionally, we used the biomass proportion of bank voles and voles of genus Microtus, for regression analysis with estimated population indices and bank vole proportion with the index of Microtus voles. In this set of analyses, we used GLMMs with a random intercept for nest ID and generalized linear models (GLMs) if the identifier, due to the number of replicates being too low, did not contribute to model performance [71][72]. We used binomial family with logistic link function to compare the proportion of prey in the diet with its annual population index value in nature the same year.…”