Informed Proposals for Local MCMC in Discrete Spaces

Zanella, Giacomo

doi:10.1080/01621459.2019.1585255

Cited by 71 publications

(117 citation statements)

References 42 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…The choice of the number of variables to select would then be related to a cost per iteration versus mixing trade‐off. See section 6.4 of Zanella () for a discussion of similar blockwise implementations. Also, computing p i ( x ) exactly may be infeasible in some contexts, and thus it would be interesting to design a version of TGS where the terms p i ( x ) are replaced by unbiased estimators while preserving the correct invariant distribution.…”

Section: Discussionmentioning

confidence: 99%

Scalable Importance Tempering and Bayesian Variable Selection

Zanella

Roberts

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

Self Cite

View full text Add to dashboard Cite

Summary We propose a Monte Carlo algorithm to sample from high dimensional probability distributions that combines Markov chain Monte Carlo and importance sampling. We provide a careful theoretical analysis, including guarantees on robustness to high dimensionality, explicit comparison with standard Markov chain Monte Carlo methods and illustrations of the potential improvements in efficiency. Simple and concrete intuition is provided for when the novel scheme is expected to outperform standard schemes. When applied to Bayesian variable‐selection problems, the novel algorithm is orders of magnitude more efficient than available alternative sampling schemes and enables fast and reliable fully Bayesian inferences with tens of thousand regressors.

show abstract

Section: Discussionmentioning

confidence: 99%

Scalable Importance Tempering and Bayesian Variable Selection

Zanella

Roberts

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

Self Cite

View full text Add to dashboard Cite

show abstract

“…The resulting proposal distributions are called globally-balanced in Zanella (2020). To understand why, consider the situation where the size of K is small and it is feasible to switch from any model to any other one, and thus we can set N(k) = K for all k.…”

Section: Specification Of the Model Proposal Distributionsmentioning

confidence: 99%

“…We recommend to set h to the identity only when it is to feasible to set N(k) = K for all k. Otherwise, it seems better to use locallybalanced proposal distributions; this is the case for instance in Zanella (2020) and our moderate-size (both in n and dimension) variable-selection example. Two choices of locally-balanced functions h are considered in Zanella (2020): h(x) = √ x and h(x) = x/(1 + x).…”

Section: Specification Of the Model Proposal Distributionsmentioning

confidence: 99%

“…The empirical results of Zanella (2020) suggest that this latter choice is superior because h is bounded, which stabilizes the normalizing constants c(k) and c(k ) and thus the acceptance probabilities. Livingstone and Zanella (2019) reached the same conclusion using locally-balanced proposal distributions for continuous random variables.…”

Section: Specification Of the Model Proposal Distributionsmentioning

confidence: 99%

See 1 more Smart Citation

Informed reversible jump algorithms

Gagnon

2021

Electron. J. Statist.

View full text Add to dashboard Cite

Incorporating information about the target distribution in proposal mechanisms generally improves Markov chain Monte Carlo algorithms. For instance, it has proved successful to incorporate gradient information in fixed-dimensional algorithms such as Hamiltonian Monte Carlo. In trans-dimensional algorithms, Green (2003) recommended to sample the parameter proposals during model switches from normal distributions with informative means and covariance matrices. These proposal distributions can be viewed as asymptotic approximations to the parameter distributions, where the limit is with regard to the sample size. Models are typically proposed using uninformed uniform distributions. In this paper, we build on the approach of Zanella (2020) for discrete spaces to incorporate information about neighbouring models. We rely on approximations to posterior model probabilities that are asymptotically exact. We prove that, in some scenarios, the samplers combining this approach with that of Green (2003) behave like ideal ones that use the exact model probabilities and sample from the correct parameter distributions, in the large-sample regime. We show that the implementation of the proposed samplers is straightforward in some cases. The methodology is applied to a real-data example. The code is available online. ‡

show abstract

“…This result is achieved by leveraging the efficiency of an approximate multi-object filtering method. The use of MCMC in discrete spaces is discussed more generally in Zanella (2019). MCMC has also been used in conjunction with, or as a replacement of, sequential Monte Carlo in the context of filtering for single-object systems (Gilks and Berzuini 2001) and multi-object systems (Khan et al 2005;Septier et al 2009;Carmi et al 2012;Maroulas and Stinis 2012;Bao and Maroulas 2017); however, this type of approach is less directly related to the method proposed in this article.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty modelling and computational aspects of data association

2021

View full text Add to dashboard Cite

A novel solution to the smoothing problem for multi-object dynamical systems is proposed and evaluated. The systems of interest contain an unknown and varying number of dynamical objects that are partially observed under noisy and corrupted observations. In order to account for the lack of information about the different aspects of this type of complex system, an alternative representation of uncertainty based on possibility theory is considered. It is shown how analogues of usual concepts such as Markov chains and hidden Markov models (HMMs) can be introduced in this context. In particular, the considered statistical model for multiple dynamical objects can be formulated as a hierarchical model consisting of conditionally independent HMMs. This structure is leveraged to propose an efficient method in the context of Markov chain Monte Carlo (MCMC) by relying on an approximate solution to the corresponding filtering problem, in a similar fashion to particle MCMC. This approach is shown to outperform existing algorithms in a range of scenarios.

show abstract

Informed Proposals for Local MCMC in Discrete Spaces

Cited by 71 publications

References 42 publications

Scalable Importance Tempering and Bayesian Variable Selection

Scalable Importance Tempering and Bayesian Variable Selection

Informed reversible jump algorithms

Uncertainty modelling and computational aspects of data association

Contact Info

Product

Resources

About