Comparison of model building and selection strategies

Abstract

One challenge an analyst often encounters when dealing with complex mark-recapture models is how to limit the number of a priori models. While all possible combinations of model structures on the different parameters (e.g., /, p) can be considered, such a strategy often results in a burdensome number of models, leading to the use of ad hoc strategies to reduce the number of models constructed. For the Cormack-Jolly-Seber data type, one example of an ad hoc strategy is to hold a general / model structure constant while investigating model structures on p, and then to hold the resulting best structure on p constant and investigate structures on /. Many comparable strategies exist. The effect of following ad hoc strategies on parameter estimates as well as for variable selection and whether model averaging can ameliorate any problems are unknown. By means of a simulation study, we have investigated this informational gap by comparing the allcombinations model building strategy with two ad hoc strategies and with truth, as well as considering the results of model averaging. We found that model selection strategy had little effect on parameter estimator bias and precision and that model averaging did improve bias and precision slightly. In terms of variable selection (i.e., cumulative Akaike's information criterion weights), model sets based on ad hoc strategies did not perform as well as those based on all combinations, as less important variables often had higher weights with the former than with the all possible combinations strategy. Increased sample size resulted in increased variable weights, with an infinite sample size resulting in all variable weights equaling 1 for variables with any predictive influence. Thus, the distinction between statistical importance (dependent on sample size) and biological importance must be recognized when utilizing cumulative weights. We recommend that allcombinations model strategy and model averaging be used. However, if an ad hoc strategy is relied upon to reduce the computational demand, parameter estimates will generally be comparable to the all-combinations strategy, but variable weights will not correspond to the all-combinations strategy.

Buffer zone use by mammals in a Cerrado protected area

“…For this set of models c was estimated as null or with ''local'' covariate (Doherty et al 2012). The most explanatory model or models was/were selected using the Akaike Information Criterion corrected for small-sample bias (AICc) (Burnham & Anderson 2002, MacKenzie et al 2006 and associated model weights (o i , Burnham & Anderson 2002).…”

“…The most explanatory model or models was/were selected using the Akaike Information Criterion corrected for small-sample bias (AICc) (Burnham & Anderson 2002, MacKenzie et al 2006 and associated model weights (o i , Burnham & Anderson 2002). This approach is considered the best method for achieving useful estimates of cumulative variable weights (w cum ) since it allows constructing a balanced model set (Doherty et al 2012).…”

Restored Agricultural Wetlands in central Iowa: Habitat Quality and Amphibian Response

“…Individual covariates were included in the estimation of the probabilities of survival, capture, and recapture. We ran all combinations of parameter structures (50 possible models, Table 1) and used the corrected Akaike's information criterion (AICc) for small sample sizes to determine which models best described the data (Doherty et al 2012). Because there was some uncertainty in model selection, we model averaged the estimates of survival, capture, and recapture probability, as well as population size for each of the four primary periods (Doherty et al 2012).…”

Investimento parental e papéis sexuais em opiliões com cuidado paternal exclusivo

“…With the best selected structure for ψ, and with the structure for still fixed as the additive effect between the time-related variable and individual categories, we built a new set of alternative models that considered p as being: (a) We used the 'step-down' approach described above (and first presented by [31]) to avoid the comparison of all possible models in a single analysis, i.e., (4 structures for the general model) x (6 structures for ψ) x (15 structures for p) x (15 structures for ) = 5,400 models, which would be a prohibitive, time consuming procedure and would greatly increase the possibility of spurious results [35,48]. However, it is still not clear if the order in which the structure of parameters is fixed or modeled affects the convergence of different approaches to the same best selected model [31,48].…”

“…However, it is still not clear if the order in which the structure of parameters is fixed or modeled affects the convergence of different approaches to the same best selected model [31,48]. In an attempt to avoid biased results due to our specific analytical implementation, we also performed the model selection procedure starting with a general model in which all parameters were considered constant.…”