Emulation of higher-order tensors in manifold Monte Carlo methods for Bayesian Inverse Problems

Lan, Shiwei; Bui-Thanh, Tan; Christie, Mike; Girolami, Mark

doi:10.1016/j.jcp.2015.12.032

Cited by 48 publications

(54 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Riemann manifold Langevin and Hamiltonian Monte Carlo methods [60,61] exploit the first and second order local structure of the posterior distribution and profit from more efficient gradient evaluation. The same holds for novel emulator-based sampling procedures [62] and approaches for posterior approximation [63]. By exploiting the proposed approach, rigorous Bayesian parameter estimation for models with hundreds of parameters could become a standard tool instead of an exception [64,65].…”

Section: Discussionmentioning

confidence: 85%

Scalable parameter estimation for genome-scale biochemical reaction networks

Fröhlich

Kaltenbacher

Theis

et al. 2016

Preprint

View full text Add to dashboard Cite

Mechanistic mathematical modeling of biochemical reaction networks using ordinary differential equation (ODE) models has improved our understanding of small-and medium-scale biological processes. While the same should in principle hold for large-and genome-scale processes, the computational methods for the analysis of ODE models which describe hundreds or thousands of biochemical species and reactions are missing so far. While individual simulations are feasible, the inference of the model parameters from experimental data is computationally too intensive. In this manuscript, we evaluate adjoint sensitivity analysis for parameter estimation in large scale biochemical reaction networks. We present the approach for time-discrete measurement and compare it to state-of-the-art methods used in systems and computational biology. Our comparison reveals a significantly improved computational efficiency and a superior scalability of adjoint sensitivity analysis. The computational complexity is effectively independent of the number of parameters, enabling the analysis of large-and genome-scale models. Our study of a comprehensive kinetic model of ErbB signaling shows that parameter estimation using adjoint sensitivity analysis requires a fraction of the computation time of established methods. The proposed method will facilitate mechanistic modeling of genome-scale cellular processes, as required in the age of omics. Author SummaryIn this manuscript, we introduce a scalable method for parameter estimation for genome-scale biochemical reaction networks. Mechanistic models for genome-scale biochemical reaction networks describe the behavior of thousands of chemical species using thousands of parameters. Standard methods for parameter estimation are usually computationally intractable at these scales. Adjoint sensitivity based approaches have been suggested to have superior scalability but any rigorous evaluation is lacking. We implement a toolbox for adjoint sensitivity analysis for biochemical reaction network which also supports the import of SBML models. We show by means of a set of benchmark models that adjoint sensitivity based approaches unequivocally outperform standard approaches for large-scale models and that the achieved speedup increases with respect to both the number of parameters and the number of chemical species in the model. This demonstrates the applicability of adjoint sensitivity based approaches to parameter estimation for genome-scale mechanistic model. The MATLAB toolbox implementing the developed methods is available from http://ICB-DCM.github.io/AMICI/.

show abstract

Section: Discussionmentioning

confidence: 85%

Scalable parameter estimation for genome-scale biochemical reaction networks

Fröhlich

Kaltenbacher

Theis

et al. 2016

Preprint

View full text Add to dashboard Cite

show abstract

“…Substituting K n with DR-∞-GMC Markov kernel into the SMC scheme can parallelize these MCMC algorithms to achieve further efficiency improvement to tackle Bayesian inverse problems at larger scale. DR-∞-mHMC can be improved further by e.g., surrogate methods [42,43] or grid methods [44] to reduce the burden of point-wise updating gradient and metric. Within the leap-frog steps of 'HMC' type algorithms, one can consider 'BFGS' type update as in quasi-Newton methods [45].…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Adaptive dimension reduction to accelerate infinite-dimensional geometric Markov Chain Monte Carlo

Lan

2019

Journal of Computational Physics

Self Cite

View full text Add to dashboard Cite

Bayesian inverse problems highly rely on efficient and effective inference methods for uncertainty quantification (UQ). Infinite-dimensional MCMC algorithms, directly defined on function spaces, are robust under refinement (through discretization, spectral approximation) of physical models. Recent development of this class of algorithms has started to incorporate the geometry of the posterior informed by data so that they are capable of exploring complex probability structures, as frequently arise in UQ for PDE constrained inverse problems. However, the required geometric quantities, including the Gauss-Newton Hessian operator or Fisher information metric, are usually expensive to obtain in high dimensions. On the other hand, most geometric information of the unknown parameter space in this setting is concentrated in an intrinsic finite-dimensional subspace. To mitigate the computational intensity and scale up the applications of infinite-dimensional geometric MCMC (∞-GMC), we apply geometry-informed algorithms to the intrinsic subspace to probe its complex structure, and simpler methods like preconditioned Crank-Nicolson (pCN) to its geometry-flat complementary subspace.In this work, we take advantage of dimension reduction techniques to accelerate the original ∞-GMC algorithms. More specifically, partial spectral decomposition (e.g. through randomized linear algebra) of the (prior or Gaussian-approximate posterior) covariance operator is used to identify certain number of principal eigen-directions as a basis for the intrinsic subspace. The combination of dimension-independent algorithms, geometric information, and dimension reduction yields more efficient implementation, (adaptive) dimensionreduced infinite-dimensional geometric MCMC. With a small amount of computational overhead, we can achieve over 70 times speed-up compared to pCN using a simulated elliptic inverse problem and an inverse problem involving turbulent combustion with thousands of dimensions after discretization. A number of error bounds comparing various MCMC proposals are presented to predict the asymptotic behavior of the proposed dimension-reduced algorithms.Gaussian. In particular, infinite-dimensional geometric MCMC (∞-GMC) [10] put a series of 'dimensionindependent' MCMC algorithms in the context of increasingly adopting geometry (gradient, Hessian). With the help of such geometric information, [10] show that with the prior-based splitting strategy, ∞-GMC algorithms can achieve up to two orders of magnitude speed up in sampling efficiency compared to vanilla pCN. However, fully computing the required geometric quantities is prohibitive in the discretized parameter space with thousands of dimensions. Therefore, it is natural to consider approximations to the gradient vector and Hessian (Fisher) matrix and compute them in a subspace with reduced dimensions. The key to the dimension reduction in this setting is to identify an intrinsic low-dimensional subspace and apply geometric methods to effectively explore its complex structure; while s...

show abstract

“…The second approach relies on a surrogate function, the gradient of which is less expensive to calculate. [11] [7] used Gaussian process (GP) to produce satisfactory results in lower dimensions. However, training a GP is itself computationally expensive and training points must be chosen with great care.…”

Section: Introductionmentioning

confidence: 99%

Neural network gradient Hamiltonian Monte Carlo

Holbrook

Shahbaba

et al. 2019

Comput Stat

View full text Add to dashboard Cite

Hamiltonian Monte Carlo is a widely used algorithm for sampling from posterior distributions of complex Bayesian models. It can efficiently explore high-dimensional parameter spaces guided by simulated Hamiltonian flows. However, the algorithm requires repeated gradient calculations, and these computations become increasingly burdensome as data sets scale. We present a method to substantially reduce the computation burden by using a neural network to approximate the gradient. First, we prove that the proposed method still maintains convergence to the true distribution though the approximated gradient no longer comes from a Hamiltonian system. Second, we conduct experiments on synthetic examples and real data to validate the proposed method. arXiv:1711.05307v2 [stat.CO]

show abstract

Emulation of higher-order tensors in manifold Monte Carlo methods for Bayesian Inverse Problems

Cited by 48 publications

References 69 publications

Scalable parameter estimation for genome-scale biochemical reaction networks

Scalable parameter estimation for genome-scale biochemical reaction networks

Adaptive dimension reduction to accelerate infinite-dimensional geometric Markov Chain Monte Carlo

Neural network gradient Hamiltonian Monte Carlo

Contact Info

Product

Resources

About