Adaptive Sequential Posterior Simulators for Massively Parallel Computing Environments

Abstract

Massively parallel desktop computing capabilities now well within the reach of individual academics modify the environment for posterior simulation in fundamental and potentially quite advantageous ways. But to fully exploit these benefits algorithms that conform to parallel computing environments are needed. Sequential Monte Carlo comes very close to this ideal whereas other approaches like Markov chain Monte Carlo do not. This paper presents a sequential posterior simulator well suited to this computing environment. The simulator makes fewer analytical and programming demands on investigators, and is faster, more reliable and more complete than conventional posterior simulators. The paper extends existing sequential Monte Carlo methods and theory to provide a thorough and practical foundation for sequential posterior simulation that is well suited to massively parallel computing environments. It provides detailed recommendations on implementation, yielding an algorithm that requires only code for simulation from the prior and evaluation of prior and data densities and works well in a variety of applications representative of serious empirical work in economics and finance. The algorithm is robust to pathological posterior distributions, generates accurate marginal likelihood approximations, and provides estimates of numerical standard error and relative numerical efficiency intrinsically. The paper concludes with an application that illustrates the potential of these simulators for applied Bayesian inference.

Sequentially Adaptive Bayesian Learning for a Nonlinear Model of the Secular and Cyclical Behavior of US Real GDP

“…Durham and Geweke [3] shows that bounded likelihood and existence of the relevant prior moment together sufficient for ergodicity. In all posterior simulators the assessment of numerical accuracy is based on a central limit theorem, which in this context takes the form…”

“…Durham and Geweke [3] deals with this issue by creating the two-pass variant of the algorithm. The first pass is exactly as described in this section, with the addition that the kernels k ( ) and transitions dQ ( ) are saved.…”

“…More generally, SABL can be used to attack a wide variety of optimization problems in economics. 3. SABL enables Bayesians to use almost any prior distribution and to modify conventional distribution by imposing constraints (illustrated here) or mixing with discrete distributions (not illustrated).…”

“…For full development of the immediate theory, refer to [2][3][4]. For the underlying probability theory the single best self-contained reference is [5].…”

Parallel Sequential Monte Carlo for Efficient Density Combination: TheDeCoMATLABToolbox

“…There are a few papers that deal with this interesting topic, see Suchard, Holmes, and West (2010), Morozov and Mathur (2012), Aldrich, Fernández-Villaverde, Gallant, and Ramırez (2011), Durham and Geweke (2014), Dziubinski and Grassi (2014) and Creel, Mandal, and Zubair (2012). This is odd given the fact that parallel computing in economics has a long history.…”