Data driven governing equations approximation using deep neural networks

Abstract

We present a numerical framework for approximating unknown governing equations using observation data and deep neural networks (DNN). In particular, we propose to use residual network (ResNet) as the basic building block for equation approximation. We demonstrate that the ResNet block can be considered as a one-step method that is exact in temporal integration. We then present two multi-step methods, recurrent ResNet (RT-ResNet) method and recursive ReNet (RS-ResNet) method. The RT-ResNet is a multi-step method on uniform time steps, whereas the RS-ResNet is an adaptive multi-step method using variable time steps. All three methods presented here are based on integral form of the underlying dynamical system. As a result, they do not require time derivative data for equation recovery and can cope with relatively coarsely distributed trajectory data. Several numerical examples are presented to demonstrate the performance of the methods. Key words. Deep neural network, residual network, recurrent neural network, governing equation discovery 1. Introduction. Recently there has been a growing interest in discovering governing equations numerically using observational data. Earlier efforts include methods using symbolic regression ([5, 43]), equation-free modeling [24], heterogeneous multiscale method (HMM) ([15]), artificial neural networks ([19]), nonlinear regression ([50]), empirical dynamic modeling ([46, 53]), nonlinear Laplacian spectral analysis ([18]), automated inference of dynamics ([44, 12, 13]), etc. More recent efforts start to cast the problem into a function approximation problem, where the unknown governing equations are treated as target functions relating the data for the state variables and their time derivatives. The majority of the methods employ certain sparsitypromoting algorithms to create parsimonious models from a large set of dictionary for all possible models, so that the true dynamics could be recovered exactly ([47]). Many studies have been conducted to effectively deal with noises in data ([7, 40]), corruptions in data ([48]), partial differential equations [38,41], etc. Methods have also been developed in conjunction with model selection approach ([28]), Koopman theory ([6]), and Gaussian process regression ([35]), to name a few. A more recent work resorts to the more traditional means of approximation by using orthogonal polynomials ([52]). The approach seeks accurate numerical approximation to the underlying governing equations, instead of their exact recovery. By doing so, many existing results in polynomial approximation theory can be applied, particularly those on sampling strategies. It was shown in [52] that data from a large number of short bursts of trajectories are more effective for equation recovery than those from a single long trajectory.On the other hand, artificial neural network (ANN), and particularly deep neural network (DNN), has seen tremendous successes in many different disciplines. The number of publications is too large to mention. Here we cite only a few relati...

Data-driven deep learning of partial differential equations in modal space

“…While many of the existing equation learning methods seek to directly approximate or learn the specific form of the governing equations, we adopt a different framework, which seeks to approximate evolution operator of the underlying equations. Such an approach was presented and analyzed in [17] for recovery of ODEs. For the PDE recovery problem considered in this paper, our first task is to reduce the problem from infinite dimension to finite dimension.…”

“…Its recovery allows one to conduct prediction of the underlying PDE and is effectively equivalent to the recovery of the equation. This is an extension of the equation recovery work from [17], where the flow map of the underlying unknown dynamical system is the goal of recovery. Unlike the ODE systems considered in [17], PDE systems, which is the focus of this paper, are of infinite dimension.…”

“…This is an extension of the equation recovery work from [17], where the flow map of the underlying unknown dynamical system is the goal of recovery. Unlike the ODE systems considered in [17], PDE systems, which is the focus of this paper, are of infinite dimension. In order to cope with infinite dimension, our method first reduces the problem into finite dimensions by utilizing a properly chosen modal space, i.e., generalized Fourier space.…”

“…The equation recovery task is then transformed into recovery of the generalized Fourier coefficients, which follow a finite dimensional dynamical system. The approximation of the finite dimensional evolution operator of the reduced system is then carried out by using deep neural network, particularly the residual network (ResNet) which has been shown to be particularly suitable for this task [17]. One of the advantages of the proposed method is that, by focusing on evolution operator, it eliminates the need for time derivatives data of the state variables.…”

Deciphering gene regulation from gene expression dynamics using deep neural network

“…Over the past year, various ideas of simulating dynamical systems with DNNs have been proposed. Attentions of those researchers are mainly focused on the concepts itself, constructing model architectures informed of basic physics laws or dealing with sophisticated spatial differential operators(42)(43)(44), etc. Algorithms proposed in these literatures were typically demonstrated on synthesized data rather than real-world situations.…”