Sparse Deep Neural Network for Nonlinear Partial Differential Equations

Abstract

More competent learning models are demanded for data processing due to increasingly greater amounts of data available in applications. Data that we encounter often have certain embedded sparsity structures. That is, if they are represented in an appropriate basis, their energies can concentrate on a small number of basis functions. This paper is devoted to a numerical study of adaptive approximation of solutions of nonlinear partial differential equations whose solutions may have singularities, by deep neural networks (DNNs) with a sparse regularization with multiple parameters. Noting that DNNs have an intrinsic multi-scale structure which is favorable for adaptive representation of functions, by employing a penalty with multiple parameters, we develop DNNs with a multi-scale sparse regularization (SDNN) for effectively representing functions having certain singularities. We then apply the proposed SDNN to numerical solutions of the Burgers equation and the Schrödinger equation. Numerical examples confirm that solutions generated by the proposed SDNN are sparse and accurate.

Machine Learning Algorithm for the Monge-Ampère Equation with Transport Boundary Conditions

“…An increasing number of algorithms have emerged in recent years for solving PDEs using neural networks [9,18,20,21,27,29,44]. Our algorithm also uses neural networks to solve the problem.…”

PI-VEGAN: Physics Informed Variational Embedding Generative Adversarial Networks for Stochastic Differential Equations

“…In recent years, the use of deep learning to solve fundamental partial differential equations (PDEs) has gained considerable attention [10,24,26], thanks to the high expressiveness of neural networks and the rapid growth of computing hardware. Among them, physics-informed neural networks (PINNs) [5,11,16,17,19,23,[27][28][29]31] are a particularly interesting approach. PINNs incorporate physical knowledge as soft constraints in the empirical loss function and employ machine learning methodologies like automatic differentiation and stochastic optimization to train the model.…”