Model averaging and muddled multimodel inferences

Abstract

Three flawed practices associated with model averaging coefficients for predictor variables in regression models commonly occur when making multimodel inferences in analyses of ecological data. Model-averaged regression coefficients based on Akaike information criterion (AIC) weights have been recommended for addressing model uncertainty but they are not valid, interpretable estimates of partial effects for individual predictors when there is multicollinearity among the predictor variables. Multicollinearity implies that the scaling of units in the denominators of the regression coefficients may change across models such that neither the parameters nor their estimates have common scales, therefore averaging them makes no sense. The associated sums of AIC model weights recommended to assess relative importance of individual predictors are really a measure of relative importance of models, with little information about contributions by individual predictors compared to other measures of relative importance based on effects size or variance reduction. Sometimes the model-averaged regression coefficients for predictor variables are incorrectly used to make model-averaged predictions of the response variable when the models are not linear in the parameters. I demonstrate the issues with the first two practices using the college grade point average example extensively analyzed by Burnham and Anderson. I show how partial standard deviations of the predictor variables can be used to detect changing scales of their estimates with multicollinearity. Standardizing estimates based on partial standard deviations for their variables can be used to make the scaling of the estimates commensurate across models, a necessary but not sufficient condition for model averaging of the estimates to be sensible. A unimodal distribution of estimates and valid interpretation of individual parameters are additional requisite conditions. The standardized estimates or equivalently the t statistics on unstandardized estimates also can be used to provide more informative measures of relative importance than sums of AIC weights. Finally, I illustrate how seriously compromised statistical interpretations and predictions can be for all three of these flawed practices by critiquing their use in a recent species distribution modeling technique developed for predicting Greater Sage-Grouse (Centrocercus urophasianus) distribution in Colorado, USA. These model averaging issues are common in other ecological literature and ought to be discontinued if we are to make effective scientific contributions to ecological knowledge and conservation of natural resources.

Demographic response of Piping Plovers suggests that engineered habitat restoration is no match for natural riverine processes

“…Unless otherwise stated, and to account for multiple competing models, we obtained model-averaged parameter estimates and unconditional standard errors for all real (e.g., survival, fidelity, detection rates) parameters (Burnham and Anderson 2002). For beta regression coefficients, we provide estimates from the top-ranked model (Cade 2015). When interpreting the difference between individual estimates, we used several types of evidence, including model ranking, the size of the estimate relative to the standard error, model weights, and confidence intervals.…”

“…From 2005 to 2009, research focused on evaluating demography and movement on new, 'engineered' sandbar habitat relative to 'natural' sandbar habitat that was deposited by high flows from 1996 to 1997 (Catlin 2009, Catlin et al 2011b, 2015. From 2012 to 2014, we studied how Piping Plovers responded to the flood-created sandbar habitat.…”

Response of Bird Populations To Long-Term Changes In Local Vegetation and Regional Forest Cover

“…For each species or group of species, we ran models with every possible subset of independent variables (n ¼ 7). We then used multi-model (Cade 2015). Importance values for each independent variable are often calculated by summing the weights of individual models that contain that variable, but recent analyses suggests that these values poorly represent the actual contribution of independent variables (Galipaud et al 2014).…”

Clearing islands as refugia for black-fronted tern (Chlidonias albostriatus) breeding colonies in braided rivers

“…We ranked all models for each analysis using Akaike's Information Criterion corrected for small sample size (AICC; Burnham & Anderson 2003). As our focus was on the particular effect of vegetation on islands, we included vegetation in all our candidate models and did not average across multiple models (Cade 2015;Banner & Higgs 2016). All analyses were carried out in R v. 3.3.3 (R Core Team 2017).…”