Bayesian adaptation of chaos representations using variational inference and sampling on geodesics

Abstract

A novel approach is presented for constructing polynomial chaos representations of scalar quantities of interest (QoI) that extends previously developed methods for adaptation in Homogeneous Chaos spaces. In this work, we develop a Bayesian formulation of the problem that characterizes the posterior distributions of the series coefficients and the adaptation rotation matrix acting on the Gaussian input variables. The adaptation matrix is thus construed as a new parameter of the map from input to QoI, estimated through Bayesian inference. For the computation of the coefficients' posterior distribution, we use a variational inference approach that approximates the posterior with a member of the same exponential family as the prior, such that it minimizes a Kullback–Leibler criterion. On the other hand, the posterior distribution of the rotation matrix is explored by employing a Geodesic Monte Carlo sampling approach, consisting of a variation of the Hamiltonian Monte Carlo algorithm for embedded manifolds, in our case, the Stiefel manifold of orthonormal matrices. The performance of our method is demonstrated through a series of numerical examples, including the problem of multiphase flow in heterogeneous porous media.

Probabilistic learning on manifolds (PLoM) with partition

“…which is estimated using the n MC realizations {[𝜼 i,𝓁 ar ], 𝓁 = 1, … , n MC }. Equation ( 7) contains the information defined by Equation (8). Indeed it is easy to verify that we have the relation…”

“…The probabilistic learning on manifolds (PLoM) method was proposed in 2016 1 as a complementary approach to existing methods in machine learning for sampling underlying distributions on manifolds. [2][3][4][5][6][7][8][9] It allows for solving unsupervised and supervised problems under uncertainty for which the training sets are small. This situation is encountered in many problems of physics and engineering sciences with expensive function evaluations.…”

“…2 and 𝜁 i = ||[𝜂 i d ]||2 . Therefore, Equation(8) can be rewritten asd 2 i,N (m i,o ) = E{Ξ i }∕𝜁 i . If ∀i ∈ {1, … , n p } we have Ξ i ≥ 𝜀 𝜁 i a.s, then ||[H 𝜀||[𝜂 d ]|| 2 a.s, that is to say ||[H wg,N m o ] − [𝜂 d ]|| 2 ∕||[𝜂 d ]|| 2 ≥ 𝜀 a.s. (iii) Using result (ii), it can be deduced that Proba{∩ n p i=1 {Ξ i ∕𝜁 i ≥ 𝜀}} = Proba{||[H wg,N m o ] − [𝜂 d ]|| 2 ∕||[𝜂 d ]|| 2 ≥ 𝜀}.…”

Compressive sensing adaptation for polynomial chaos expansions

“…Furthermore, techniques for sampling from distributions defined on Stiefel manifolds as in [9] would enable estimation of posterior distributions of projection matrices and would be another step towards the ultimate goal that is the fully Bayesian solution of the problem. Recent progress on this direction has shown that an approach for sampling from the marginal posteriors of the coefficients and the rotation matrix, using a combination of variational inference and hamiltonian Monte Carlo on the Stiefel manifold, can result in an efficient way of exploring the joint posterior [55], though more generic approaches are yet to be developed. Here we seek to derive the gradient of the 2 error function…”

Compressed Principal Component Analysis of Non-Gaussian Vectors

“…The use of the PCE for constructing a parameterized representation of a non-Gaussian random field that models the parameter of a boundary value problem, in order to identify it solving a statistical inverse problem has been initiated in [13,18,14] and used and revisited in [11,23,10,42,35,36,8,34,45] for statistical inverse problems in low or in high stochastic dimension. For the statistical inverse identification of the coefficients of the PCE, the Bayesian approach has been proposed in [18,32,2,43,38,29,50,41,56].…”