An auxiliary variable method for Langevin based MCMC algorithms

IEEE Trans. Signal Process.

Benazza-Benyahia

et al. 2020

Self Cite

The dimension and the complexity of inference problems have dramatically increased in statistical signal processing. It thus becomes mandatory to design improved proposal schemes in Metropolis-Hastings algorithms, providing large proposal transitions that are accepted with high probability. The proposal density should ideally provide an accurate approximation to the target density with a low computational cost. In this paper, we derive a novel Metropolis-Hastings proposal, inspired from Langevin dynamics, where the drift term is preconditioned by an adaptive matrix constructed through a Majorization-Minimization strategy. We propose several variants of lowcomplexity curvature metrics applicable to large scale problems. We demonstrate the geometric ergodicity of the resulting chain for the class of super-exponential distributions. The proposed method is shown to exhibit a good performance in two signal recovery examples.

Section: B Multispectral Image Denoising With a Multivariate Priormentioning

confidence: 99%

Majorize–Minimize Adapted Metropolis–Hastings Algorithm

IEEE Trans. Signal Process.

Benazza-Benyahia

et al. 2020

Self Cite

“…Therefore, it is desirable to separate heterogeneous matrices in order to facilitate sampling. This has been successfully achieved using DA strategies [8], [9], [10]. Specifically, auxiliary variables u ∈ R P , are added to the model 2 with a predefined joint distribution with density q(x, u).…”

Section: A Principlementioning

confidence: 99%

“…Thereby, when implemented through a Gibbs sampler, they may turn out to be less efficient; in particular, when G depends on some target parameters evolving along the algorithm. Recently, new sampling strategies have been proposed as alternatives to optimization based Gaussian sampling [8], [9], [10]. By adding some auxiliary variables, the authors demonstrate, in several inverse problems applications, that sampling becomes much easier in the new augmented space.…”

Section: Introductionmentioning

confidence: 99%

A Data Augmentation Approach for Sampling Gaussian Models in High Dimension

Abboud

2019 27th European Signal Processing Conference (EUSIPCO)

et al. 2019

Recently, methods based on Data Augmentation (DA) strategies have shown their efficiency for dealing with highdimensional Gaussian sampling within Gibbs samplers compared to iterative-based sampling (e.g., Perturbation-Optimization). However, they are limited by the feasibility of the direct sampling of the auxiliary variable. This paper reviews DA sampling algorithms for Gaussian sampling and proposes a DA method which is especially useful when direct sampling of the auxiliary variable is not straightforward from a computational viewpoint. Experiments in two vibration analysis applications show the good performance of the proposed algorithm.

“…This formulation has been widely used in energy-minimization approaches [87][88][89] where the initial optimization problem is replaced by the minimization of the constructed surrogate function. Furthermore, this technique has been recently extended to sampling algorithms [90]. The initial intractable posterior distribution to sample from is replaced by the conditional distribution of the target signal given the auxiliary variables.…”

Section: Likelihoodmentioning

confidence: 99%

A Variational Bayesian Approach for Image Restoration—Application to Image Deblurring With Poisson–Gaussian Noise

Zheng

IEEE Trans. Comput. Imaging

et al. 2017

Self Cite

In this paper, a methodology is investigated for signal recovery in the presence of non-Gaussian noise. In contrast with regularized minimization approaches often adopted in the literature, in our algorithm the regularization parameter is reliably estimated from the observations. As the posterior density of the unknown parameters is analytically intractable, the estimation problem is derived in a variational Bayesian framework where the goal is to provide a good approximation to the posterior distribution in order to compute posterior mean estimates. Moreover, a majorization technique is employed to circumvent the difficulties raised by the intricate forms of the non-Gaussian likelihood and of the prior density. We demonstrate the potential of the proposed approach through comparisons with state-of-the-art techniques that are specifically tailored to signal recovery in the presence of mixed Poisson-Gaussian noise. Results show that the proposed approach is efficient and achieves performance comparable with other methods where the regularization parameter is manually tuned from the ground truth.