Fence methods for mixed model selection

Abstract

Many model search strategies involve trading off model fit with model complexity in a penalized goodness of fit measure. Asymptotic properties for these types of procedures in settings like linear regression and ARMA time series have been studied, but these do not naturally extend to nonstandard situations such as mixed effects models, where simple definition of the sample size is not meaningful. This paper introduces a new class of strategies, known as fence methods, for mixed model selection, which includes linear and generalized linear mixed models. The idea involves a procedure to isolate a subgroup of what are known as correct models (of which the optimal model is a member). This is accomplished by constructing a statistical fence, or barrier, to carefully eliminate incorrect models. Once the fence is constructed, the optimal model is selected from among those within the fence according to a criterion which can be made flexible. In addition, we propose two variations of the fence. The first is a stepwise procedure to handle situations of many predictors; the second is an adaptive approach for choosing a tuning constant. We give sufficient conditions for consistency of fence and its variations, a desirable property for a good model selection procedure. The methods are illustrated through simulation studies and real data analysis.Comment: Published in at http://dx.doi.org/10.1214/07-AOS517 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

The Fence Methods

“…These concerns, such as the above, led to the development of a new class of strategies for model selection, known as the fence methods, first introduced by Jiang et al [13]. Also see Jiang et al [14].…”

“…In a way, this is similar to one of the difficulties with the information criteria noted in the previous section. Jiang et al [13] came up with an idea, known as adaptive fence (AF), to let the data "speak" on how to choose this cut-off. Let M denote the set of candidate models.…”

“…This means that the optimal model is neither f nor * . We refer the (technical) details on how to handle these extreme cases to Jiang et al [13].…”

“…In the original paper of Jiang et al [13], the fence inequality (2) was presented with the on the right side replaced bŷ,̃, wherê,̃is an estimated standard deviation of the left side. Although, in some special cases, such as when is the negative log-likelihood,̂,̃is easy to obtain, the computation of̂,̃, in general, can be timeconsuming.…”

A Parametric Test to Discriminate Between a Linear Regression Model and a Linear Latent Growth Model

“…These approaches are based on the choice of models that minimize an estimate of a specific criterion which usually involves a trade-off between the closeness of the fit to the data and the complexity of the model. We refer to the paper of Muller et al (2013) for a review of these approaches and other methods such as shrinkage methods like the LASSO (Tibshirani, 1996), Fence methods (Jiang et al, 2008) and Bayesian methods. The validity of all the methods proposed depends on the underlying assumptions.…”