Adaptive deep density approximation for Fokker-Planck equations

Tang, Kejun; Wan, Xiaoliang; Liao, Qifeng

doi:10.1016/j.jcp.2022.111080

Cited by 21 publications

(10 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Looking more closely, the large errors are concentrated around the location of singularity of u, i.e., the curve of active constraints {x : u(x(µ)) = µ}, except for the case µ = 20 where the inequality constraint is nonactive, keeping the smoothness of the optimal control function. Note that adaptive sampling strategies [51,50,13] may be used to improve the accuracy in the singularity region, which will be left for future study.…”

Section: Testmentioning

confidence: 99%

AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems

Peng¹,

Xiao²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Parametric optimal control problems governed by partial differential equations (PDEs) are widely found in scientific and engineering applications. Traditional grid-based numerical methods for such problems generally require repeated solutions of PDEs with different parameter settings, which is computationally prohibitive especially for problems with high-dimensional parameter spaces. Although recently proposed neural network methods make it possible to obtain the optimal solutions simultaneously for different parameters, challenges still remain when dealing with problems with complex constraints. In this paper, we propose AONN, an adjoint-oriented neural network method, to overcome the limitations of existing approaches in solving parametric optimal control problems. In AONN, the neural networks are served as parametric surrogate models for the control, adjoint and state functions to get the optimal solutions all at once. In order to reduce the training difficulty and handle complex constraints, we introduce an iterative training framework inspired by the classical direct-adjoint looping (DAL) method so that penalty terms arising from the Karush-Kuhn-Tucker (KKT) system can be avoided. Once the training is done, parameter-specific optimal solutions can be quickly computed through the forward propagation of the neural networks, which may be further used for analyzing the parametric properties of the optimal solutions. The validity and efficiency of AONN is demonstrated through a series of numerical experiments with problems involving various types of parameters.

show abstract

Section: Testmentioning

confidence: 99%

AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems

Peng¹,

Xiao²,

Tang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…KRnet provides a more expressive density model than real NVP for the same model size. More details about KRnet can be found in [23,24].…”

Section: Krnet Mapmentioning

confidence: 99%

“…A certain structure needs to be intro-duced to determine a map. Some typical structures include the Knothe-Rosenblatt (K-R) rearrangement [19], neural ODE [20] and the composition of many simple maps used in flow-based deep generative models such as NICE [21], real NVP [22], KRnet [23,24,25], to name a few.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dimension-reduced KRnet maps for high-dimensional Bayesian inverse problems

Yu¹,

Tang²,

Wan³

et al. 2023

Preprint

View full text Add to dashboard Cite

We present a dimension-reduced KRnet map approach (DR-KRnet) for high-dimensional Bayesian inverse problems, which is based on an explicit construction of a map that pushes forward the prior measure to the posterior measure in the latent space. Our approach consists of two main components: data-driven VAE prior and density approximation of the posterior of the latent variable. In reality, it may not be trivial to initialize a prior distribution that is consistent with available prior data; in other words, the complex prior information is often beyond simple hand-crafted priors. We employ variational autoencoder (VAE) to approximate the underlying distribution of the prior dataset, which is achieved through a latent variable and a decoder. Using the decoder provided by the VAE prior, we reformulate the problem in a low-dimensional latent space. In particular, we seek an invertible transport map given by KRnet to approximate the posterior distribution of the latent variable. Moreover, an efficient physics-constrained surrogate model without any labeled data is constructed to reduce the computational cost of solving both forward and adjoint problems involved in likelihood computation. With numerical experiments, we demonstrate the accuracy and efficiency of DR-KRnet for high-dimensional Bayesian inverse problems.

show abstract

“…While ELM emerged nearly two decades ago, the investigation of this technique for the numerical solution of differential equations has appeared only quite recently, alongside the proliferation of deep neural network (DNN) based PDE solvers in the past few years (see e.g. [50,47,16,61,22,8,30,57,55,11,38,53,34,56], among many others). In [62,52,36,37] the ELM technique has been used for solving linear ordinary or partial differential equations (ODEs/PDEs) with single hidden-layer feedforward neural networks, in which certain polynomials (e.g.…”

Section: Introductionmentioning

confidence: 99%

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Ni¹,

Dong²

2022

Preprint

View full text Add to dashboard Cite

The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. We show that HLConcFNNs have the remarkable property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers, the approximation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works.

show abstract

Adaptive deep density approximation for Fokker-Planck equations

Cited by 21 publications

References 38 publications

AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems

AONN: An adjoint-oriented neural network method for all-at-once solutions of parametric optimal control problems

Dimension-reduced KRnet maps for high-dimensional Bayesian inverse problems

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Contact Info

Product

Resources

About