Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control

Abstract

Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of the nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world datasets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900 to 1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control.

Characterization of Acoustic Emissions From Analogue Rocks Using Sparse Regression‐DMDc

“…Currently there are several emerging methods that allow for a broader viewpoint of building models directly from noisy data, recent innovations in DMD, for instance, have shown that statistical bagging methods can greatly increase the discovery of robust, accurate and stable linear models (Askham & Kutz, 2018; Sashidhar & Kutz, 2021). This has motivated the use of ensembling and bagging for building nonlinear, parsimonious dynamic models (Fasel et al., 2022; Hirsh et al., 2021). Such non‐linear dynamic models have the potential to unravel the contemporaneously overlapping information that are representative of AE or seismic events generated by different processes, for example, fracturing, slip on faults, chemical reactions, or the various phenomena that transpire during fluid movement through porous media.…”

Discovery of nonlinear dynamical systems using a Runge–Kutta inspired dictionary-based sparse regression approach

“…But the question remains as to whether the algorithm will converge to the correct sparse solution. A remedy to this can be to use the rationale of an ensemble, proposed in [43] to build an ensemble of sparse models. It can provide statistical quantities for the feature candidates in the dictionary.…”

Learning Anisotropic Interaction Rules from Individual Trajectories in a Heterogeneous Cellular Population

“…The former intrinsically produces models with high bias and low variance, while the latter produces models with low bias and high variance. Such pooling strategies have been recently explored in [22], where it is found that identifying a single model can be improved by pooling models learned from subsets of the data. However, this has not been extended to classifying the data itself into species, and finding a model for each species.…”

“…where |C| denotes the number of elements in C. This ensemble average preserves the force modes that identify C. As explored in [22], in some cases it may be more appropriate to use the coefficient median, or take a weighted average. We leave these for future work.…”