Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression

Reinbold, Patrick A. K.; Kageorge, Logan; Schatz, Michael F.; Grigoriev, Roman O.

doi:10.1038/s41467-021-23479-0

Cited by 78 publications

(41 citation statements)

References 29 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It can also be used to jointly discovery coordinates and dynamics simultaneously [31], [32]. The integral formulation of SINDy [33] has also proven to be powerful, enabling the identification of governing equations in a weak form that averages over control volumes; this approach has recently been used to discover a hierarchy of fluid and plasma models [34]- [37]. The open source software package, PySINDy 1 , has been developed in Python to integrate the various extensions of SINDy [38].…”

Section: Introductionmentioning

confidence: 99%

A Toolkit for Data-Driven Discovery of Governing Equations in High-Noise Regimes

Delahunt

Kutz

2022

IEEE Access

View full text Add to dashboard Cite

We consider the data-driven discovery of governing equations from time-series data in the limit of high noise. The algorithms developed describe an extensive toolkit of methods for circumventing the deleterious effects of noise in the context of the sparse identification of nonlinear dynamics (SINDy) framework. We offer two primary contributions, both focused on noisy data acquired from a system ẋ = f (x). First, we propose, for use in high-noise settings, an extensive toolkit of critically enabling extensions for the SINDy regression method, to progressively cull functionals from an over-complete library and yield a set of sparse equations that regress to the derivate ẋ. This toolkit includes: (regression step) weight timepoints based on estimated noise, use ensembles to estimate coefficients, and regress using FFTs; (culling step) leverage linear dependence of functionals, and restore and protect culled functionals based on Figures of Merit (FoMs). In a novel Assessment step, we define FoMs that compare model predictions to the original time-series (i.e., x(t) rather than ẋ(t)). These innovations can extract sparse governing equations and coefficients from high-noise time-series data (e.g., 300% added noise). For example, it discovers the correct sparse libraries in the Lorenz system, with median coefficient estimate errors equal to 1%−3% (for 50% noise), 6%−8% (for 100% noise); and 23%−25% (for 300% noise). The enabling modules in the toolkit are combined into a single method, but the individual modules can be tactically applied in other equation discovery methods (SINDy or not) to improve results on high-noise data. Second, we propose a technique, applicable to any model discovery method based on ẋ = f (x), to assess the accuracy of a discovered model in the context of non-unique solutions due to noisy data. Currently, this non-uniqueness can obscure a discovered model's accuracy and thus a discovery method's effectiveness. We describe a technique that uses linear dependencies among functionals to transform a discovered model into an equivalent form that is closest to the true model, enabling more accurate assessment of a discovered model's accuracy.

show abstract

Section: Introductionmentioning

confidence: 99%

A Toolkit for Data-Driven Discovery of Governing Equations in High-Noise Regimes

Delahunt

Kutz

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A potential example use-case is in phase identification, where the latent variables may fall into disjoint clusters depending on the phase attributed to the input. Additionally, more complex [36,38] or physically-motivated [39,40] approaches to symbolic regression may produce more robust and meaningful symbolic expressions to describe the latent space, including the ability to produce a full Pareto frontier [36] of increasingly complex but accurate formula. We leave these improvements to future work.…”

Section: Discussionmentioning

confidence: 99%

Machine learning of Kondo physics using variational autoencoders and symbolic regression

Miles,

Carbone,

Sturm

et al. 2021

Preprint

View full text Add to dashboard Cite

We employ variational autoencoders to extract physical insight from a dataset of one-particle Anderson impurity model spectral functions. Autoencoders are trained to find a low-dimensional, latent space representation that faithfully characterizes each element of the training set, as measured by a reconstruction error. Variational autoencoders, a probabilistic generalization of standard autoencoders, further condition the learned latent space to promote highly interpretable features. In our study, we find that the learned latent space components strongly correlate with well known, but nontrivial, parameters that characterize emergent behaviors in the Anderson impurity model. In particular, one latent space component correlates with particle-hole asymmetry, while another is in near one-to-one correspondence with the Kondo temperature, a dynamically generated low-energy scale in the impurity model. With symbolic regression, we model this component as a function of bare physical input parameters and "rediscover" the non-perturbative formula for the Kondo temperature. The machine learning pipeline we develop opens opportunities to discover new domain knowledge in other physical systems.

show abstract

“…In [15], authors proposed two-phase bi-objective symbolic regression method and discussed how to choose the model that fit the training data as precisely as possible and is consistent with the prior knowledge about the system given in the form of nonlinear inequality and equality constraints. In [16], the authors, using SR, reconstruct the pressure and the forcing field for a weakly turbulent fluid flow only when the velocity field is known.…”

Section: Methods and Modelsmentioning

confidence: 99%

Data Analysis and Symbolic Regression Models for Predicting CO and NOx Emissions from Gas Turbines

Kochueva

Nikolskii

2021

Computation

View full text Add to dashboard Cite

Predictive emission monitoring systems (PEMS) are software solutions for the validation and supplementation of costly continuous emission monitoring systems for natural gas electrical generation turbines. The basis of PEMS is that of predictive models trained on past data to estimate emission components. The gas turbine process dataset from the University of California at Irvine open data repository has initiated a challenge of sorts to investigate the quality of models of various machine learning methods to build a model for predicting CO and NOx emissions depending on ambient variables and the parameters of the technological process. The novelty and features of this paper are: (i) a contribution to the study of the features of the open dataset on CO and NOx emissions for gas turbines, which will enable one to more objectively compare different machine learning methods for further research; (ii) for the first time for the CO and NOx emissions, a model based on symbolic regression and a genetic algorithm is presented—the advantage of this being the transparency of the influence of factors and the interpretability of the model; (iii) a new classification model based on the symbolic regression model and fuzzy inference system is proposed. The coefficients of determination of the developed models are: R2=0.83 for NOx emissions, R2=0.89 for CO emissions.

show abstract

Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression

Cited by 78 publications

References 29 publications

A Toolkit for Data-Driven Discovery of Governing Equations in High-Noise Regimes

A Toolkit for Data-Driven Discovery of Governing Equations in High-Noise Regimes

Machine learning of Kondo physics using variational autoencoders and symbolic regression

Data Analysis and Symbolic Regression Models for Predicting CO and NOx Emissions from Gas Turbines

Contact Info

Product

Resources

About