High-Dimensional Uncertainty Quantification via Active and Rank-Adaptive Tensor Regression

He, Zichang; Zhang, Zheng

doi:10.1109/epeps48591.2020.9231388

Cited by 8 publications

(6 citation statements)

References 45 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…47 Dimension-reducing surrogate methods leverage ideas from Gaussian process (GP) regression, 48,49 active subspaces, [50][51][52] kernel-PCA, [53][54][55] polynomial chaos expansions (PCEs), 56 and other numerical schemes based on low-rank tensor decomposition. [57][58][59][60][61] Deep learning motivates another class of methods in dimensionality reduction. Hinton and Salakhutdinov proposed autoencoders to learn a latent representation of training examples in a data-driven way.…”

Section: High-dimensional Uncertainty Qualificationmentioning

confidence: 99%

Probabilistic partition of unity networks for high‐dimensional regression problems

Fan

Trask

D’Elia

et al. 2023

Numerical Meth Engineering

View full text Add to dashboard Cite

We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly paralleliazable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. We also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

show abstract

Section: High-dimensional Uncertainty Qualificationmentioning

confidence: 99%

Probabilistic partition of unity networks for high‐dimensional regression problems

Fan

Trask

D’Elia

et al. 2023

Numerical Meth Engineering

View full text Add to dashboard Cite

show abstract

“…A framework based on kernel-PCA in conjunction with Kriging or PCE is proposed by Lataniotis et al [73]. Low-rank tensor-based schemes have been also proposed to to address the issue of high dimensional input [74,75].…”

Section: Dimension Reduction For High-dimensional Uqmentioning

confidence: 99%

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Kontolati,

Loukrezis,

Giovanis

et al. 2022

Preprint

View full text Add to dashboard Cite

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10 ≥2 ) stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The "curse of dimensionality" can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

show abstract

“…46 Dimension-reducing surrogate methods leverage ideas from Gaussian process (GP) regression 47,48 ; active subspaces 49,50,51 ; kernel-PCA 52,53,54 ; polynomial chaos expansions (PCEs) 55 ; and other numerical schemes based on low-rank tensor decomposition. 56,57,58,59,60 Deep learning motivates another class of methods in dimensionality reduction. Hinton and Salakhutdinov proposed autoencoders to learn a latent representation of training examples in a data-driven way.…”

Section: High-dimensional Uncertainty Qualificationmentioning

confidence: 99%

Probabilistic partition of unity networks for high-dimensional regression problems

Fan¹,

Trask²,

D’Elia³

et al. 2022

Preprint

View full text Add to dashboard Cite

We explore the probabilistic partition of unity network (PPOU-Net) model 1,2 in the context of high-dimensional regression problems. With the PPOU-Nets, the target function for any given input is approximated by a mixture of experts model, where each cluster is associated with a fixed-degree polynomial. The weights of the clusters are determined by a DNN that defines a partition of unity. The weighted average of the polynomials approximates the target function and produces uncertainty quantification naturally. Our training strategy leverages automatic differentiation and the expectation maximization (EM) algorithm. During the training, we (i) apply gradient descent to update the DNN coefficients; (ii) update the polynomial coefficients using weighted least-squares solves; and (iii) compute the variance of each cluster according to a closed-form formula derived from the EM algorithm. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. We also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

show abstract

High-Dimensional Uncertainty Quantification via Active and Rank-Adaptive Tensor Regression

Cited by 8 publications

References 45 publications

Probabilistic partition of unity networks for high‐dimensional regression problems

Probabilistic partition of unity networks for high‐dimensional regression problems

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Probabilistic partition of unity networks for high-dimensional regression problems

Contact Info

Product

Resources

About