Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Hobert, James P.; Jung, Yeun Ji; Khare, Kshitij; Qin, Qian

doi:10.1111/sjos.12310

Cited by 8 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We shall refer to h as a mixing density . Heavy‐tailed error densities can be produced by choosing h with appropriate behavior near the origin . Some typical choices for h are the gamma, inverse gamma, generalized inverse Gaussian, and log‐normal densities.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A note on the convergence rate of MCMC for robust Bayesian multivariate linear regression with proper priors

Backlund

Hobert

2020

Comp and Math Methods

Self Cite

View full text Add to dashboard Cite

The multivariate linear regression model with errors from a scale mixture of Gaussian densities yields a complex likelihood function. Combining this likelihood with any nontrivial prior distribution leads to a highly intractable posterior density. If a conditionally conjugate prior is used, then there is a well known and easy‐to‐implement data augmentation (DA) algorithm available for exploring the posterior. Hobert et al recently showed that, under an improper conditionally conjugate prior (and weak regularity conditions), the Markov chain that drives the DA algorithm converges at a geometric rate. Unfortunately, the model studied by Hobert et al can only be used in situations where the X matrix has full column rank. In this note, analogous convergence rate results are established for a proper conditionally conjugate prior. An important advantage of using a proper prior is that, not only is the X matrix allowed to be column rank deficient, but it can also have more columns than rows, that is, our model is applicable in cases where p>n. This is an important extension in the era of big data.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Heavy-tailed error densities can be produced by choosing h with appropriate behavior near the origin. [1][2][3] Some typical choices for h are the gamma, inverse gamma, generalized inverse Gaussian, and log-normal densities. However, in principle, h can be taken to be any density on the positive half-line.…”

Section: Introductionmentioning

confidence: 99%

A note on the convergence rate of MCMC for robust Bayesian multivariate linear regression with proper priors

Backlund

Hobert

2020

Comp and Math Methods

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is particularly true for the so-called Data Augmentation (DA) algorithm, which is a widely used technique for constructing Markov chains by introducing unobserved/latent random variables. In this context, often, (a) the transition density can only be expressed as an intractable high-dimensional integral, and/or (b) the stationary density is only available up to an unknown normalizing constant 1 , see Albert and Chib (1993); ; Roy (2012); Polson, Scott, and Windle (2013); Choi and Hobert (2013); Hobert et al (2015); Qin and Hobert (2016); Pal et al (2017) to name just a few.…”

Section: Introductionmentioning

confidence: 99%

Consistent estimation of the spectrum of trace class Data Augmentation algorithms

Chakraborty¹,

Khare²

2019

Bernoulli

Self Cite

View full text Add to dashboard Cite

Markov chain Monte Carlo is widely used in a variety of scientific applications to generate approximate samples from intractable distributions. A thorough understanding of the convergence and mixing properties of these Markov chains can be obtained by studying the spectrum of the associated Markov operator. While several methods to bound/estimate the second largest eigenvalue are available in the literature, very few general techniques for consistent estimation of the entire spectrum have been proposed. Existing methods for this purpose require the Markov transition density to be available in closed form, which is often not true in practice, especially in modern statistical applications. In this paper, we propose a novel method to consistently estimate the entire spectrum of a general class of Markov chains arising from a popular and widely used statistical approach known as Data Augmentation. The transition densities of these Markov chains can often only be expressed as intractable integrals. We illustrate the applicability of our method using real and simulated data.MSC 2010 subject classifications: Primary 60J22; secondary 62F15.Chakraborty and Khare/Spectrum estimation for trace class DA algorithms 2 Carlo (MCMC) methods. Here, a Markov chain X = (X n ) n≥0 with equilibrium probability distribution Π is generated (using any standard MCMC strategies such as Metropolis Hastings, Gibbs sampler etc.) and then a Monte Carlo average based on those Markov chain realizations is used to estimate πg.If the Markov chain X = (X n ) n≥0 is Harris ergodic (which is the case if the corresponding Markov transition density is strictly positive everywhere), then the cumulative averages based on the Markov chain realizations consistently estimate the integral of interest (see Asmussen and Glynn (2011)). The accuracy of the estimate depends on two factors: (a) the convergence behavior of the Markov chain to its stationary distribution, and (b) the dependence between the successive realizations of the chain at stationarity. An operator theoretic framework provides a unified way of analyzing these two related factors. Let us consider the Hilbert space L 2 (π) of real valued functions f with finite second moment with respect to π. This is a

show abstract

“…Remark 1. Conditions (S1) & (S2) are known to be necessary for posterior propriety (Fernández and Steel, 1999;Hobert et al, 2016).…”

Section: Introductionmentioning

confidence: 99%

“…There is a well-known data augmentation (DA) algorithm that can be used to explore this intractable density (Liu, 1996). Hobert et al (2016) (hereafter HJK&Q) performed convergence rate analyses of the Markov chains underlying this DA algorithm and an alternative Haar PX-DA algorithm. In this paper, we provide a substantial improvement of HJK&Q's main result.…”

Section: Introductionmentioning

confidence: 99%

Trace-class Monte Carlo Markov chains for Bayesian multivariate linear regression with non-Gaussian errors

Qin

Hobert

2018

Journal of Multivariate Analysis

Self Cite

View full text Add to dashboard Cite

Let π denote the intractable posterior density that results when the likelihood from a multivariate linear regression model with errors from a scale mixture of normals is combined with the standard non-informative prior. There is a simple data augmentation algorithm (based on latent data from the mixing density) that can be used to explore π. Let h(·) and d denote the mixing density and the dimension of the regression model, respectively. Hobert et al. (2016) have recently shown that, if h converges to 0 at the origin at an appropriate rate, and ∞ 0 u d 2 h(u) du < ∞, then the Markov chains underlying the DA algorithm and an alternative Haar PX-DA algorithm are both geometrically ergodic. In fact, something much stronger than geometric ergodicity often holds. Indeed, it is shown in this paper that, under simple conditions on h, the Markov operators defined by the DA and Haar PX-DA Markov chains are trace-class, i.e., compact with summable eigenvalues. Many of the mixing densities that satisfy Hobert et al.'s (2016) conditions also satisfy the new conditions developed in this paper. Thus, for this set of mixing densities, the new results provide a substantial strengthening of Hobert et al.'s (2016) conclusion without any additional assumptions. For example, Hobert et al. (2016) showed that the DA and Haar PX-DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log-normal, Fréchet (with shape parameter larger than d/2), or inverted gamma (with shape parameter larger than d/2). The results in this paper show that, in each of these cases, the DA and Haar PX-DA Markov operators are, in fact, trace-class.

show abstract

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Cited by 8 publications

References 31 publications

A note on the convergence rate of MCMC for robust Bayesian multivariate linear regression with proper priors

A note on the convergence rate of MCMC for robust Bayesian multivariate linear regression with proper priors

Consistent estimation of the spectrum of trace class Data Augmentation algorithms

Trace-class Monte Carlo Markov chains for Bayesian multivariate linear regression with non-Gaussian errors

Contact Info

Product

Resources

About