Gibbs sampler and coordinate ascent variational inference: A set-theoretical review

Bayesian Nonlinear Models for Repeated Measurement Data: An Overview, Implementation, and Applications

“…4. As for the Bayesian sampling algorithms, combinations of Gibbs sampler [39], Metropolis-Hastings algorithm [40], Hamiltonian Monte Carlo [41], and No-U-Turn sampler [42] are popularly used, among many others [43][44][45]. We explain these in detail in Section 6.…”

“…Estimation objective Maximize a likelihood [14,15,21] Sample from a posterior [22,28,30] Computation algorithm First-order approximation [36], Laplace approximation [37], and stochastic approximation of EM algorithm [38] Gibbs sampler [39], Metropolis-Hastings algorithm [40], Hamiltonian…”

“…We resort to MCMC technique [174] to sample from the full joint density π(Θ, σ 2 , α, B, Ω|y 1:N ) (16). Among many MCMC techniques, we use the Gibbs sampling algorithm [39,168] to exploit the conditional independence [148] induced by the hierarchical formulation. A generic Gibbs sampler would cycle in turn through each of the conditional distributions for the parameter blocks Θ, σ 2 , α, B, and Ω as follows:…”

“…In practice, as similar to DIC (38), WAIC (39) is obtained by stochastic approximations. Given posterior samples {(Ψ) (s) } S s=1 ∼ π(Ψ|y 1:N ), the LPPD and p W terms may be approximated by…”

“…, where only the mean function f i (t i , θ i ) differs across the H candidate models, and second, approximate LPPD and p W by using a MCMC method, and finally, obtain an approximation of WAIC (39) corresponding to each model.…”