Vector Operations for Accelerating Expensive Bayesian Computations – A Tutorial Guide

Abstract

Many applications in Bayesian statistics are extremely computationally intensive. However, they are often inherently parallel, making them prime targets for modern massively parallel processors. Multi-core and distributed computing is widely applied in the Bayesian community, however, very little attention has been given to fine-grain parallelisation using single instruction multiple data (SIMD) operations that are available on most modern CPUs. In this work, we practically demonstrate, using standard programming libraries, the utility of the SIMD approach for several topical Bayesian applications. Using the C programming language, we show that SIMD can improve the single-core floating point arithmetic performance by up to a factor of 6× compared scalar C code and more than 25× compared with optimised R code. Such improvements are multiplicative to any gains achieved through multi-core processing. We illustrate the potential of SIMD for accelerating Bayesian computations and provide the reader with techniques for exploiting modern massively parallel processing environments.

Bayesian uncertainty quantification to identify population level vaccine hesitancy behaviours

“…Furthermore various approaches that exploit surrogate or approximate models have been shown to dramatically improve performance whilst sacrificing minimal accuracy (Bon et al, 2022;Everitt and Rowińska, 2020;Prangle, 2016;Baker, 2020, 2021;Jasra et al, 2019;Warne et al, 2022aWarne et al, ,b, 2018. Finally, state-of-the-art massively parallel computing hardware has potential to complement algorithmic advances to enable real-time analysis (Hurn et al, 2016;Kulkarni et al, , 2022Lee et al, 2010;Mahani and Sharabiani, 2015;Warne et al, 2022c).…”

Multifidelity multilevel Monte Carlo to accelerate approximate Bayesian parameter inference for partially observed stochastic processes

“…In particular, one substantial advantage of utilising MF-MLMC-ABC based upon rejection sampling, is that there few synchronisation steps. Therefore, most of the stochastic simulations (especially the approximate simulations) can exploit many parallel computing architectures, such as general-purpose graphics processing units (GPGPUs) [111,112,113], single instruction multiple data (SIMD) CPU processors [114,115], and recent advances in AI hardware [116]. This leads to methods in which the statistical efficiency also directly scales to state-of-the-art massively parallel computing.…”