Efficient Spectral Sparse Grid Methods and Applications to High-Dimensional Elliptic Equations II. Unbounded Domains

Shen, Jie; Yu, Haijun

doi:10.1137/110834950

Cited by 38 publications

(37 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, high-dimensional hyperbolic cross spectral projection is hard to calculate. In practice, we use efficient sparse grid spectral transforms developed in [33] and [34] to approximate the projection. After a numerical network is built, one may further train it to obtain a network function that is more accurate than the sparse grid interpolation.…”

Section: Error Bounds Of Approximating Some High-dimensional Smooth Fmentioning

confidence: 99%

Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

Li¹

2020

CiCP

View full text Add to dashboard Cite

Deep neural network with rectified linear units (ReLU) is getting more and more popular recently. However, the derivatives of the function represented by a ReLU network are not continuous, which limit the usage of ReLU network to situations only when smoothness is not required. In this paper, we construct deep neural networks with rectified power units (RePU), which can give better approximations for smooth functions. Optimal algorithms are proposed to explicitly build neural networks with sparsely connected RePUs, which we call PowerNets, to represent polynomials with no approximation error. For general smooth functions, we first project the function to their polynomial approximations, then use the proposed algorithms to construct corresponding PowerNets. Thus, the error of best polynomial approximation provides an upper bound of the best RePU network approximation error. For smooth functions in higher dimensional Sobolev spaces, we use fast spectral transforms for tensorproduct grid and sparse grid discretization to get polynomial approximations. Our constructive algorithms show clearly a close connection between spectral methods and deep neural networks: a PowerNet with n layers can exactly represent polynomials up to degree s n , where s is the power of RePUs. The proposed PowerNets have potential applications in the situations where high-accuracy is desired or smoothness is required. the curse of dimensionality for an important class of problems corresponding to compositional functions. In the general function approximation aspect, it has been proved by Yarotsky [12] that DNNs using rectified linear units (abbr. ReLU, a non-smooth activation function defined as σ 1 (x) := max{0, x}) need at most O (ε d k (log |ε| + 1)) units and nonzero weights to approximation functions in Sobolev space W k,∞ ([−1, 1] d ) within ε error. This is similar to the results of shallow networks with one hidden layer of C ∞ activation units, but only optimal up to a O (log |ε|) factor. Similar results for approximating functions in W k,p ([−1, 1] d ) with p < ∞ using ReLU DNNs are given by Petersen and Voigtlaender[13]. The significance of the works by Yarotsky [12] and Peterson and Voigtlaender [13] is that by using a very simple rectified nonlinearity, DNNs can obtain high order approximation property. Shallow networks do not hold such a good property. Other works show ReLU DNNs have high-order approximation property include the work by E and Wang [14] and the recent work by Opschoor et al. [15], the latter one relates ReLU DNNs to high-order finite element methods.A basic fact used in the error estimate given in [12] and [13] is that x 2 , x y can be approximated by a ReLU network with O (log |ε|) layers. To remove this approximation error and the extra factor O (log |ε|) in the size of neural networks, we proposed to use rectified power units (RePU) to construct exact neural network representations of polynomials [16]. The RePU function is defined aswhere s is a non-negative integer. When s = 1, we have the Heaviside step func...

show abstract

Section: Error Bounds Of Approximating Some High-dimensional Smooth Fmentioning

confidence: 99%

Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

Li¹

2020

CiCP

View full text Add to dashboard Cite

show abstract

“…As seen, the model has 12 parameters, that is, the equilibrium Michaelis constant K m,1-6 , and maximal reaction velocity V max,1-6 . In this case study, we focus on the model response e 3p , which is also a function of the input signal I(t) and the gain of negative feedback G 4 in Equation (37). Details about the biological description of the model and its parameters can be found in Reference 42.…”

Section: Example 3: Autocrine Signaling Of Living Cellsmentioning

confidence: 99%

“…It should be mentioned that techniques such as sparse grids were proposed to overcome the challenges, which requires a relatively small number of grids when the number of uncertainties is moderately large . However, as compared to SC, there are a few challenges to be addressed using the sparse grids, such as how to construct sparse grids for optimal decay behaviors, which are open questions. Compared to the SC‐based UQ, the SG‐based gPC is more accurate as previously reported in the literature .…”

Section: Introductionmentioning

confidence: 99%

Probabilistic surrogate models for uncertainty analysis: Dimension reduction‐based polynomial chaos expansion

Son

2019

Numerical Meth Engineering

View full text Add to dashboard Cite

Summary This paper presents an approach for efficient uncertainty analysis (UA) using an intrusive generalized polynomial chaos (gPC) expansion. The key step of the gPC‐based uncertainty quantification (UQ) is the stochastic Galerkin (SG) projection, which can convert a stochastic model into a set of coupled deterministic models. The SG projection generally yields a high‐dimensional integration problem with respect to the number of random variables used to describe the parametric uncertainties in a model. However, when the number of uncertainties is large and when the governing equation of the system is highly nonlinear, the SG approach‐based gPC can be challenging to derive explicit expressions for the gPC coefficients because of the low convergence in the SG projection. To tackle this challenge, we propose to use a bivariate dimension reduction method (BiDRM) in this work to approximate a high‐dimensional integral in SG projection with a few one‐ and two‐dimensional integrations. The efficiency of the proposed method is demonstrated with three different examples, including chemical reactions and cell signaling. As compared to other UA methods, such as the Monte Carlo simulations and nonintrusive stochastic collocation (SC), the proposed method shows its superior performance in terms of computational efficiency and UA accuracy.

show abstract

“…As a commonality of sparse grids, the collocation point set X i with interpolation level i is usually designed in a nested form, making X i-1 , X i , so that many collocation points will appear with increasing depth of interpolation, thus accordingly averting the repeated computation. S CGL can be an excellent candidate for grid type selection, as suggested by Shen and Yu [35] and utilised here. Construction of these nested sets for S CGL is performed as follows…”

Section: Collocation Points From Sparse Gridmentioning

confidence: 99%

Effects of wind speed probabilistic and possibilistic uncertainties on generation system adequacy

Sun

Bie

Xie³

et al. 2015

IET Generation, Transmission & Distribution

View full text Add to dashboard Cite

A random fuzzy model is proposed to express the probabilistic and possibilistic uncertainties of wind speed simultaneously. In this model, wind speed is represented by a random variable following Weibull distribution, indicating the probabilistic uncertainty. The Weibull distribution parameters of wind speed are fuzzy numbers, meaning the possibilistic uncertainty of wind speed. For estimating distribution parameters, a multi-objective optimisation problem is developed based on cumulative probability and probability distributions of wind speed. The proposed model is then combined with traditional generation system adequacy (GSA) evaluation method to investigate the effect of wind speed uncertainties on GSA. To overcome the difficulty in calculating fuzzy GSA indices, sparse grid is utilised to select collocation points and single-index regression is employed to fit the relationship between adequacy indices and wind speed parameters. This study illustrates the effectiveness of the model from its application to IEEE Modified Reliability Test System. Compared with previous researches, the proposed model is suitable for the case of incomplete data or containing some outliers. It provides more helpful interval information on wind speed and adequacy indices.

show abstract

Efficient Spectral Sparse Grid Methods and Applications to High-Dimensional Elliptic Equations II. Unbounded Domains

Cited by 38 publications

References 17 publications

Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units

Probabilistic surrogate models for uncertainty analysis: Dimension reduction‐based polynomial chaos expansion

Effects of wind speed probabilistic and possibilistic uncertainties on generation system adequacy

Contact Info

Product

Resources

About