On Model Selection Consistency of Bayesian Method for Normal Linear Models

Bayesian Methods for High Dimensional Linear Models

“…Here also we denote the number of explanatory variables as P n which is possibly much larger than the sample size n and it is assumed that the regression coefficients satisfy the sparseness condition $\underset{n\to \infty }{\lim }{\sum }_{j=1}^{P_n}\mid {{\beta }_j}^{\ast }\mid <\infty$, where β j * is the j th ‘true’ regression coefficient. The sparseness condition describes a general situation when all explanatory variables are relevant, but most of them have very small effects [96]. Several authors have studied the asymptotic properties of Bayesian variable selection methods under different regularity conditions [68,96-99].…”

“…The sparseness condition describes a general situation when all explanatory variables are relevant, but most of them have very small effects [96]. Several authors have studied the asymptotic properties of Bayesian variable selection methods under different regularity conditions [68,96-99]. Wang et al [96], Jiang [97] and Jiang [98] used a framework developed by Wasserman [100], for showing consistency in the context of density estimation.…”

“…Several authors have studied the asymptotic properties of Bayesian variable selection methods under different regularity conditions [68,96-99]. Wang et al [96], Jiang [97] and Jiang [98] used a framework developed by Wasserman [100], for showing consistency in the context of density estimation. According to Wasserman [100], ‘density consistency’ is defined as ‘given a proper prior to propose joint densities f ( x , y ) of response y and explanatory variables vector x , the posterior-proposed densities are often close to the true density for large n ’.…”

“…Jiang [98] defined a proper prior for different sets of explanatory variables to prove that the posterior-proposed densities with different parameterizations were consistent to the true density. Wang et al [96] further proved the ‘regression function consistency’, which ensures good performance of high dimensional Bayesian variable selection methods, especially when some regression coefficients are bounded away from zero, while the rest are exactly zero. Jiang [97] also studied the convergence rates of the fitted densities for generalized linear models and established consistency under some realistic assumptions.…”

“…Jiang [97] also studied the convergence rates of the fitted densities for generalized linear models and established consistency under some realistic assumptions. In summary, the asymptotic results reveal that with appropriate prior specification, high dimensional Bayesian variable selection methods not only can identify the ‘true’ model with probability 1, but also can give consistent parameter estimates [96], facilitating tremendous applications in a variety of fields. For other asymptotic results refer Casella et al [19] or Moreno et al [101].…”