Ecologists overestimate the importance of predictor variables in model averaging: a plea for cautious interpretations

Abstract

Summary1. Information-theory procedures are powerful tools for multimodel inference and are now standard methods in ecology. When performing model averaging on a given set of models, the importance of a predictor variable is commonly estimated by summing the weights of models where the variable appears, the so-called sum of weights (SW). However, SWs have received little methodological attention and are frequently misinterpreted. 2. We assessed the reliability of SW by performing model selection and averaging on simulated data sets including variables strongly and weakly correlated to the response variable and a variable unrelated to the response. Our aim was to investigate how useful SWs are to inform about the relative importance of predictor variables. 3. SW can take a wide range of possible values, even for predictor variables unrelated to the response. As a consequence, SW with intermediate values cannot be confidently interpreted as denoting importance for the considered predictor variable. Increasing sample size using an alternative information criterion for model selection or using only a subset of candidate models for model averaging did not qualitatively change our results: a variable of a given effect size can take a wide range of SW values. 4.Contrary to what is assumed in many ecological studies, it seems hazardous to define a threshold for SW above which a variable is considered as having a statistical effect on the response and SW is not a measure of effect size. Although we did not consider every possible condition of analysis, it is likely that in most situations, SW is a poor estimate of variable's importance.

Oviposition site selection of a threatened moth Eriogaster catax (Lepidoptera: Lasiocampidae) in agricultural landscape—implications for its conservation

“…To quantify the relative variable importance of each explanatory variable we used the sum of Akaike weights (SW) for all models (Burnham and Anderson 2002;Giam and Olden 2016, but see;Galipaud et al 2014;Cade 2015). In addition, we used hierarchical partitioning to calculate an estimate of the independent effect of each explanatory factor present in the best models on response variables (Chevan and Sutherland 1991).…”

Information to Act: Household Characteristics are Predictors of Domestic Infestation with the Chagas Vector Triatoma dimidiata in Central America

“…Therefore, RI should not be viewed as an indication of statistical effect, but as a measure of how certain we can be that the variable is present in the "best approximating model with increasing sample size," and it should not be interpreted if the model does not fit the data. 40 We report descriptive statistics of the variables measured in the villages and present statistics for the four models fitted to the data, the results of the likelihood ratio tests, and prediction statistics of the models. We also report on the results of multimodel evaluations: lowest AIC, number of generations before convergence, highest Akaike weights, and prediction statistics for the weighted model.…”

Associations between changing climate and body condition over decades in two southern hemisphere passerine birds

“…We report only top models within ΔAICc of 2 units of the best supported model (see for full model output Additional file 2: Table S1 and Additional file 3: Table S2). Confidence intervals of individual predictor variables were used as indicators of parameter importance [44]. All continuous, explanatory variables were centered around their means and divided by their SD's (z-scores) before including them in the models.…”