A Data Augmentation Approach for Sampling Gaussian Models in High Dimension

Abstract

Recently, methods based on Data Augmentation (DA) strategies have shown their efficiency for dealing with highdimensional Gaussian sampling within Gibbs samplers compared to iterative-based sampling (e.g., Perturbation-Optimization). However, they are limited by the feasibility of the direct sampling of the auxiliary variable. This paper reviews DA sampling algorithms for Gaussian sampling and proposes a DA method which is especially useful when direct sampling of the auxiliary variable is not straightforward from a computational viewpoint. Experiments in two vibration analysis applications show the good performance of the proposed algorithm.

On the Use of Structured Prior Models for Bayesian Compressive Sensing of Modulated Signals

“…The sampling step of u from (A5) can be easily done since HH = I M (nonregular sampling matrix) [58]. For more general sensing matrices, for example when H is a Gaussian sampling matrix, one can still sample efficiently the auxiliary variable following [68]. Note the auxiliary variable trick improves the computational complexity of one iteration of the Gibbs sampler but may induce high correlation between successive samples.…”

High-dimensional Gaussian sampling: a review and a unifying approach based on a stochastic proximal point algorithm

“…By capitalizing on possible specific structures of {Q i } i∈ [2] , it may be desirable to separate Q 1 and Q 2 in two different hopefully simpler steps of a Gibbs sampler. To this purpose, this section discusses data augmentation (DA) approaches which introduce one (or several) auxiliary variable u ∈ R k such that the joint distribution of the couple (θ, u) yields simple conditional distributions thus sampling steps within a Gibbs sampler [8,60,61,99]. Then a straightforward marginalization of the auxiliary variable u permits to retrieve the distribution π, either exactly or in an asymptotic regime depending on the nature of the DA scheme.…”

“…and yields proper marginal distributions π(θ) and π(u). Figure 4 describes the directed acyclic graphs (DAG) associated with two hierarchical models proposed in [60,61] to decouple Q 1 from Q 2 by involving auxiliary variables. In the following, we detail the motivations behind these two data augmentation schemes.…”

“…Again if Q 1 presents a nice structure, the specific approaches reviewed in subsection 2.2 can be employed. If this is not the case, the authors in [60,61] proposed a generalization of EDA, called GEDA, to simplify the whole Gibbs sampling procedure when Q arises from a hierarchical Bayesian model. In such models, Q 1 (and a fortiori Q 2 ) naturally admits an explicit decomposition which writes…”

“…Again, this approach has the great advantage of decoupling the two precision matrices Q 1 and Q 2 defining Q since they are not simultaneously involved in any of the two steps of the Gibbs sampler. In [60], the authors showed that exact DA schemes (i.e., EDA and GEDA) generally outperform AXDA as far as Gaussian sampling is concerned. This was expected since the AXDA framework proposed is not specifically designed for Gaussian targets but for a wide family of distributions.…”