Summary.We consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined. Using an information theoretic argument we derive a measure p D for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest. In general p D approximately corresponds to the trace of the product of Fisher's information and the posterior covariance, which in normal models is the trace of the 'hat' matrix projecting observations onto fitted values. Its properties in exponential families are explored. The posterior mean deviance is suggested as a Bayesian measure of fit or adequacy, and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages. Adding p D to the posterior mean deviance gives a deviance information criterion for comparing models, which is related to other information criteria and has an approximate decision theoretic justification. The procedure is illustrated in some examples, and comparisons are drawn with alternative Bayesian and classical proposals. Throughout it is emphasized that the quantities required are trivial to compute in a Markov chain Monte Carlo analysis.

Summary The essentials of our paper of 2002 are briefly summarized and compared with other criteria for model comparison. After some comments on the paper's reception and influence, we consider criticisms and proposals forimprovement made by us and others.

Model comparison is discussed from an information theoretic point of view. In particular the posterior predictive entropy is related to the target yielding DIC and modifications thereof. The adequacy of criteria for posterior predictive model comparison is also investigated depending on the comparison to be made. In particular variable selection as a special problem of model choice is formalized in different ways according to whether the comparison is a comparison across models or within an encompassing model and whether a joint or conditional sampling scheme is applied. DIC has been devised for comparisons across models. Its use in variable selection and that of other criteria is illustrated for a simulated data set.

This article addresses the problem of formally defining the 'effective number of parameters' in a Bayesian model which is assumed to be given by a sampling distribution and a prior distribution for the parameters. The problem occurs in the derivation of information criteria for model comparison which often trade off 'goodness of fit' and 'model complexity'. It also arises in (frequentist) attempts to estimate the error variance in regression models with informative priors on the regression coefficients, for example, in smoothing. It is argued that model complexity can be conceptualized as a feature of the joint distribution of the observed variables and the random parameters and might be formally described by a measure of dependence. The universal and accurate estimation of terms of model complexity is a challenging problem in practice. Several well-known criteria for model comparison are interpreted and discussed along these lines.

No abstract