Trace-class Monte Carlo Markov chains for Bayesian multivariate linear regression with non-Gaussian errors

Qin, Qian; Hobert, James P.

doi:10.1016/j.jmva.2018.03.012

Cited by 7 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, Jung & Hobert () showed that, when d =1 and the mixing density is inverted Gamma with shape parameter larger than 1/2, the Markov operator associated with the DA Markov chain is a trace‐class operator, which implies that the corresponding chain converges at a geometric rate. (Qin & Hobert () provide a substantial generalization of the results in Jung & Hobert (). )…”

Section: Introductionmentioning

confidence: 58%

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Hobert

Jung

Khare

et al. 2018

Scandinavian J Statistics

Self Cite

View full text Add to dashboard Cite

When Gaussian errors are inappropriate in a multivariate linear regression setting, it is often assumed that the errors are iid from a distribution that is a scale mixture of multivariate normals. Combining this robust regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX-DA algorithm that can be used to explore this posterior. This paper provides conditions (on the mixing density) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo algorithms. Letting d denote the dimension of the response, the main result shows that the DA and Haar PX-DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log-normal, inverted Gamma (with shape parameter larger than d=2) or Fréchet (with shape parameter larger than d=2). The results also apply to certain subsets of the Gamma, F and Weibull families.We assume throughout the paper that .N1/ and .N 2/ hold. Under these two conditions, the Markov chain of interest is well-defined, and we can engage in a convergence rate analysis

show abstract

Section: Introductionmentioning

confidence: 58%

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Hobert

Jung

Khare

et al. 2018

Scandinavian J Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…While compact operators were once thought to be rare in MCMC problems with uncountable state spaces (Chan and Geyer, 1994), a string of recent results suggests that trace-class DA Markov operators are not at all rare (see e.g. Qin and Hobert, 2018;Chakraborty and Khare, 2017;Choi and Román, 2017;Pal et al, 2017). Furthermore, by exploiting a simple trick, we are able to broaden the applicability of our method well beyond DA algorithms.…”

Section: Introductionmentioning

confidence: 92%

“…Moreover, when h(·) is a standard pdf on R + , these univariate densities are often members of a standard parametric family. The following proposition about the resulting DA operator is proved in Qin and Hobert (2018).…”

Section: Bayesian Linear Regression Model With Non-gaussian Errorsmentioning

confidence: 99%

See 1 more Smart Citation

Estimating the spectral gap of a trace-class Markov operator

Qin¹,

Hobert²,

Khare³

2019

Electron. J. Statist.

Self Cite

View full text Add to dashboard Cite

The utility of a Markov chain Monte Carlo algorithm is, in large part, determined by the size of the spectral gap of the corresponding Markov operator. However, calculating (and even approximating) the spectral gaps of practical Monte Carlo Markov chains in statistics has proven to be an extremely difficult and often insurmountable task, especially when these chains move on continuous state spaces.In this paper, a method for accurate estimation of the spectral gap is developed for general state space Markov chains whose operators are non-negative and trace-class. The method is based on the fact that the second largest eigenvalue (and hence the spectral gap) of such operators can be bounded above and below by simple functions of the power sums of the eigenvalues. These power sums often have nice integral representations. A classical Monte Carlo method is proposed to estimate these integrals, and a simple sufficient condition for finite variance is provided. This leads to asymptotically valid confidence intervals for the second largest eigenvalue (and the spectral gap) of the Markov operator. In contrast with previously existing techniques, our method is not based on a near-stationary version of the Markov chain, which, paradoxically, cannot be obtained in a principled manner without bounds on the spectral gap. On the other hand, it can be quite expensive from a computational standpoint. The efficiency of the method is studied both theoretically and empirically. want to approximate the integralare the first m elements of a well-behaved Markov chain with stationary density π(·). Unlike classical Monte Carlo estimators,Ĵ m is not based on iid random elements. Indeed, the elements of the chain are typically neither identically distributed nor independent. Given var π f, the variance of f (·) under the stationary distribution, the accuracy ofĴ m is primarily determined by two factors: (i) the convergence rate of the Markov chain, and (ii) the correlation between the f (Φ k )s when the chain is stationary. These two factors are related, and can be analyzed jointly under an operator theoretic framework.The starting point of the operator theoretic approach is the Hilbert space of functions that are square integrable with respect to the target pdf, π(·). The Markov transition function that gives rise to Φ = {Φ k } ∞ k=0 defines a linear (Markov) operator on this Hilbert space. (Formal definitions are given in Section 2.) If Φ is reversible, then it is geometrically ergodic if and only if the corresponding Markov operator admits a positive spectral gap (Roberts and Rosenthal , 1997;Kontoyiannis and Meyn, 2012). The gap, which is a real number in (0, 1], plays a fundamental role in determining the mixing properties of the Markov chain, with larger values corresponding to better performance. For instance, suppose Φ 0 has pdf π 0 (·) such that dπ 0 /dπ is in the Hilbert space, and let d(Φ k ; π) denote the total variation distance between the distribution of Φ k and the chain's stationary distribution. Then, if δ denotes the s...

show abstract

“…This is particularly true for the so-called Data Augmentation (DA) algorithm, which is a widely used technique for constructing Markov chains by introducing unobserved/latent random variables. In this context, often, (a) the transition density can only be expressed as an intractable high-dimensional integral, and/or (b) the stationary density is only available up to an unknown normalizing constant 1 , see Albert and Chib (1993); ; Roy (2012); Polson, Scott, and Windle (2013); Choi and Hobert (2013); Hobert et al (2015); Qin and Hobert (2016); Pal et al (2017) to name just a few.…”

Section: Introductionmentioning

confidence: 99%

Consistent estimation of the spectrum of trace class Data Augmentation algorithms

Chakraborty¹,

Khare²

2019

Bernoulli

View full text Add to dashboard Cite

Markov chain Monte Carlo is widely used in a variety of scientific applications to generate approximate samples from intractable distributions. A thorough understanding of the convergence and mixing properties of these Markov chains can be obtained by studying the spectrum of the associated Markov operator. While several methods to bound/estimate the second largest eigenvalue are available in the literature, very few general techniques for consistent estimation of the entire spectrum have been proposed. Existing methods for this purpose require the Markov transition density to be available in closed form, which is often not true in practice, especially in modern statistical applications. In this paper, we propose a novel method to consistently estimate the entire spectrum of a general class of Markov chains arising from a popular and widely used statistical approach known as Data Augmentation. The transition densities of these Markov chains can often only be expressed as intractable integrals. We illustrate the applicability of our method using real and simulated data.MSC 2010 subject classifications: Primary 60J22; secondary 62F15.Chakraborty and Khare/Spectrum estimation for trace class DA algorithms 2 Carlo (MCMC) methods. Here, a Markov chain X = (X n ) n≥0 with equilibrium probability distribution Π is generated (using any standard MCMC strategies such as Metropolis Hastings, Gibbs sampler etc.) and then a Monte Carlo average based on those Markov chain realizations is used to estimate πg.If the Markov chain X = (X n ) n≥0 is Harris ergodic (which is the case if the corresponding Markov transition density is strictly positive everywhere), then the cumulative averages based on the Markov chain realizations consistently estimate the integral of interest (see Asmussen and Glynn (2011)). The accuracy of the estimate depends on two factors: (a) the convergence behavior of the Markov chain to its stationary distribution, and (b) the dependence between the successive realizations of the chain at stationarity. An operator theoretic framework provides a unified way of analyzing these two related factors. Let us consider the Hilbert space L 2 (π) of real valued functions f with finite second moment with respect to π. This is a

show abstract

Trace-class Monte Carlo Markov chains for Bayesian multivariate linear regression with non-Gaussian errors

Cited by 7 publications

References 19 publications

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

Estimating the spectral gap of a trace-class Markov operator

Consistent estimation of the spectrum of trace class Data Augmentation algorithms

Contact Info

Product

Resources

About