Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial

Abstract

Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multipurpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.

A Computationally More Efficient Bayesian Approach for Estimating Continuous-Time Models

“…For all three Bayesian model implementations, the same 1,000 generated data sets and starting values were used. To monitor sampling and stop sampling when a certain stopping rule applies, we used the procedure described in Hecht, Gische, et al (2019) with one MCMC chain, an iteration block size of 50, a burning share of 0.10, a thinning interval of 1 (no thinning), and maximum number of iterations = 200,000. Sampling was stopped when all parameters reached an effective sample size ESS !…”

“…In the present work, we apply the approach proposed by Hecht, Gische, et al (2019) to reduce the run time of a Bayesian univariate continuous-time model for unequally spaced assessment designs. We chose to use the Bayesian software JAGS for its flexibility and stability.…”

“…To optimize the Bayesian estimation run time for this model, we use the approach described by Hecht, Gische, et al (2019). The central idea is to use the sample scatter matrix S (also called the mean-corrected sums of squares and cross product [SSCP] matrix, Carroll & Green, 1997) and the sample mean vector y instead of the raw response data.…”

“…Many approaches for run time optimization exist and could potentially be implemented in Bayesian model estimation. One promising approach comes from Hecht, Gische, Vogel, and Zitzmann (2019), who suggest reformulating the model in terms of the model implied covariance matrix and model implied mean vector (which can be easily derived using a structural equation modeling [SEM] framework), and using these for MCMC sampling. By applying this method, the information to be modeled is reduced from possibly millions of data points to one sample scatter matrix (with a Wishart distribution) and one sample mean vector (with a multivariate normal distribution).…”

Continuous-time modeling in prevention research: An illustration

“…Future research should investigate how run time of such models could be reduced. One promising approach is illustrated by Hecht, Gische, Vogel, and Zitzmann (2019).…”