Should I use fixed effects or random effects when I have fewer than five levels of a grouping factor in a mixed-effects model?

Abstract

As linear mixed-effects models (LMMs) have become a widespread tool in ecology, the need to guide the use of such tools is increasingly important. One common guideline is that one needs at least five levels of the grouping variable associated with a random effect. Having so few levels makes the estimation of the variance of random effects terms (such as ecological sites, individuals, or populations) difficult, but it need not muddy one’s ability to estimate fixed effects terms—which are often of primary interest in ecology. Here, I simulate datasets and fit simple models to show that having few random effects levels does not strongly influence the parameter estimates or uncertainty around those estimates for fixed effects terms—at least in the case presented here. Instead, the coverage probability of fixed effects estimates is sample size dependent. LMMs including low-level random effects terms may come at the expense of increased singular fits, but this did not appear to influence coverage probability or RMSE, except in low sample size (N = 30) scenarios. Thus, it may be acceptable to use fewer than five levels of random effects if one is not interested in making inferences about the random effects terms (i.e. when they are ‘nuisance’ parameters used to group non-independent data), but further work is needed to explore alternative scenarios. Given the widespread accessibility of LMMs in ecology and evolution, future simulation studies and further assessments of these statistical methods are necessary to understand the consequences both of violating and of routinely following simple guidelines.

Divergent responses of butterflies and bees to burning and grazing management in tallgrass prairies

“…We tested the likelihood ratio between models with the random effects structure of year nested within site vs. models with just site as a random effect. We found that models that included year within site differed significantly from models that included only site as a random effect, indicating that these models can parameterize temporal variation despite the grouping factor having only two levels (Gomes, 2022 ). This method accounts for the well‐documented phenomenon of interannual variation in insect pollinators (e.g., Herrera, 1988 ; Price et al, 2005 ).…”

Payables Management and Financial Value of Commercial Banks in Kenya

“…Random effect models assist in controlling for unobserved heterogeneity when the heterogeneity is constant over time and not correlated with independent variables. If the random effects assumption holds, the random effects estimator is more efficient than the fixed effects model (Gomes, 2022). The results are as shown in Table 7 below.…”

Effects of vessel traffic and ocean noise on gray whale stress hormones

“…Despite limiting models to just one random effect (i.e., whale ID) and reducing the number of random effects included in the model, singular fits persisted. Since this warning message may also relate to the variance of one or more combinations of the effects being close to zero 78 , we believe that the singular fit is due to the low sample size while splitting the data between many relevant fixed and random effects. Therefore, exclusion of relevant variables may not necessarily resolve the issue; thus, we selected models based on AIC, parsimony, and inclusion of variables relevant for explaining fGC variation (Table 1).…”