Constrained estimation using penalization and MCMC

Gallant, A. Ronald; Hong, Han; Leung, Michael P.; Li, Jessie

doi:10.1016/j.jeconom.2021.02.004

Cited by 1 publication

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The differences in the model complexity from dif-ferent domains have led to the existence of a wide range of MCMC sampling algorithms. Some of the prominent ones are Metropolis-Hastings algorithm [14]- [16], Gibbs sampling [17]- [19], Langevin MCMC [20]- [22], rejection sampling [23], [24], importance sampling [25], [26], sequential MCMC [27], adaptive MCMC [28], parallel tempering (tempered) MCMC [29]- [32], reversible-jump MCMC [33], [34], specialised MCMC methods for discrete time series models [35]- [37], constrained parameter and model settings [38], [39], and likelihood free MCMC [40]. MCMC sampling methods have also been used for data augmentation [41], [42], model fusion [43], model selection [44], [45], and interpolation [46].…”

Section: A Introductionmentioning

confidence: 99%

Bayesian Neural Networks via MCMC: A Python-Based Tutorial

Chandra,

Simmons

2024

IEEE Access

View full text Add to dashboard Cite

Bayesian inference provides a methodology for parameter estimation and uncertainty quantification in machine learning and deep learning methods. Variational inference and Markov Chain Monte-Carlo (MCMC) sampling methods are used to implement Bayesian inference. In the past three decades, MCMC sampling methods have faced some challenges in being adapted to larger models (such as in deep learning) and big data problems. Advanced proposal distributions that incorporate gradients, such as a Langevin proposal distribution, provide a means to address some of the limitations of MCMC sampling for Bayesian neural networks. Furthermore, MCMC methods have typically been constrained to statisticians and currently not well-known among deep learning researchers. We present a tutorial for MCMC methods that covers simple Bayesian linear and logistic models, and Bayesian neural networks. The aim of this tutorial is to bridge the gap between theory and implementation via coding, given a general sparsity of libraries and tutorials to this end. This tutorial provides code in Python with data and instructions that enable their use and extension. We provide results for some benchmark problems showing the strengths and weaknesses of implementing the respective Bayesian models via MCMC. We highlight the challenges in sampling multi-modal posterior distributions for the case of Bayesian neural networks and the need for further improvement of convergence diagnosis methods.INDEX TERMS MCMC; Bayesian deep learning; Bayesian neural networks; Bayesian linear regression; Bayesian inference

show abstract