Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data

Abstract

The sparse identification of nonlinear dynamics (SINDy) is a regression framework for the discovery of parsimonious dynamic models and governing equations from time-series data. As with all system identification methods, noisy measurements compromise the accuracy and robustness of the model discovery procedure. In this work we develop a variant of the SINDy algorithm that integrates automatic differentiation and recent time-stepping constrained motivated by Rudy et al. [1] for simultaneously (i) denoising the data, (ii) learning and parametrizing the noise probability distribution, and (iii) identifying the underlying parsimonious dynamical system responsible for generating the time-series data. Thus within an integrated optimization framework, noise can be separated from signal, resulting in an architecture that is approximately twice as robust to noise as state-of-the-art methods, handling as much as 40% noise on a given time-series signal and explicitly parametrizing the noise probability distribution. We demonstrate this approach on several numerical examples, from Lotka-Volterra models to the spatio-temporal Lorenz 96 model. Further, we show the method can learn a diversity of probability distributions for the measurement noise, including Gaussian, uniform, Gamma, and Rayleigh distributions.

Simultaneous Identification and Denoising of Dynamical Systems

“…Modified SINDy. Modified SINDy [21] introduces a variable N to estimate the noise, and penalizes violation of the dynamical system constraint by the denoised state. Introducing the evolution operator E t (x, C) that advances the differential equation with coefficients C from initial condition x,…”

“…Test Problems. Here we consider three common test problems the Duffing oscillator, Lorenz 63 attractor, and the Van der Pol Oscillator; see, e.g., [6,9,21]. We focus on these low-dimensional problems were we can perform Monte Carlo tests to evaluate performance over multiple realizations of noise.…”

“…Without a sparsity promoting constraint, we compare SIDDS and LSOI. With a sparsity promoting constraint, we compare SIDDS+ 0 with SINDy using STLS as implemented in PySINDy [12,23], and Modified SINDy (mSINDy) as implemented by [21]. These comparisons are shown in Figure 6.7 using the standard conditions mentioned in subsection 6.1 with additive i.i.d.…”

Discrepancy Modeling Framework: Learning missing physics, modeling systematic residuals, and disambiguating between deterministic and random effects

“…We leverage recent mathematical advancements in data-driven model discovery and evaluate the interplay between g(x, t) and N (µ, σ), showing how their relative sizes determine the ability to disambiguate deterministic from random or noisy effects. To encourage exploration and expansion of discrepancy modeling, we employ base coding packages for each model discovery method implementation [23][24][25][26][27][28][29][30][31][32][33][34][35][36]. We found that the performance characteristics of our suite of model discovery methods matched their documented performance for dynamical systems modeling.…”

An improved sparse identification of nonlinear dynamics with Akaike information criterion and group sparsity

“… The group sparsity is embedded in the sparse regression to learn the coefficients of PDEs due to the increased complexity of nonlinear dynamics. Considering the situation in which noise exists [ 53 ], input data may be polluted by noise or other perturbation elements. To avoid generating error for derivative estimation, it is required to divide the observations into noisy estimations and estimated measurement data to simultaneously denoise the measurements and to determine the probability distribution of the noise.…”