Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws

Tenachi, Wassim; Ibata, Rodrigo; Diakogiannis, Foivos I.

doi:10.48550/arxiv.2303.03192

Cited by 5 publications

(4 citation statements)

References 58 publications

(106 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are various methods available for utilizing NNs in symbolic regression for more than just feature selection, one of which is AIFeynman (Udrescu et al, 2020). While AIFeynman is based on the questionable assumption that the gradient of an NN provides useful information, a direct prediction of the equation using recurrent neural networks presents a promising avenue for improved symbolic regression (Petersen et al, 2021;Tenachi et al, 2023).…”

Section: Discussionmentioning

confidence: 99%

Data-Driven Equation Discovery of a Cloud Cover Parameterization

Grundner

Beucler

Gentine

et al. 2023

Preprint

View full text Add to dashboard Cite

A promising method for improving the representation of clouds in climate models, and hence climate projections, is to develop machine learning-based parameterizations using output from global storm-resolving models. While neural networks can achieve state-of-the-art performance, they are typically climate model-specific, require post-hoc tools for interpretation, and struggle to predict outside of their training distribution. To avoid these limitations, we combine symbolic regression, sequential feature selection, and physical constraints in a hierarchical modeling framework. This framework allows us to discover new equations diagnosing cloud cover from coarse-grained variables of global storm-resolving model simulations. These analytical equations are interpretable by construction and easily transferable to other grids or climate models. Our best equation balances performance and complexity, achieving a performance comparable to that of neural networks ($R^2=0.94$) while remaining simple (with only 13 trainable parameters). It reproduces cloud cover distributions more accurately than the Xu-Randall scheme across all cloud regimes (Hellinger distances $<0.09$), and matches neural networks in condensate-rich regimes. When applied and fine-tuned to the ERA5 reanalysis, the equation exhibits superior transferability to new data compared to all other optimal cloud cover schemes. Our findings demonstrate the effectiveness of symbolic regression in discovering interpretable, physically-consistent, and nonlinear equations to parameterize cloud cover.

show abstract

Section: Discussionmentioning

confidence: 99%

Data-Driven Equation Discovery of a Cloud Cover Parameterization

Grundner

Beucler

Gentine

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…There are various methods available for utilizing NNs in symbolic regression for more than just feature selection, one of which is AIFeynman (Udrescu et al, 2020). While AIFeynman is based on the questionable assumption that the gradient of an NN provides useful information, a direct prediction of the equation using recurrent NNs presents a promising avenue for improved symbolic regression (Petersen et al, 2021;Tenachi et al, 2023).…”

Section: Journal Of Advances In Modeling Earth Systemsmentioning

confidence: 99%

Data‐Driven Equation Discovery of a Cloud Cover Parameterization

Grundner,

Beucler,

Gentine

et al. 2024

J Adv Model Earth Syst

View full text Add to dashboard Cite

A promising method for improving the representation of clouds in climate models, and hence climate projections, is to develop machine learning‐based parameterizations using output from global storm‐resolving models. While neural networks (NNs) can achieve state‐of‐the‐art performance within their training distribution, they can make unreliable predictions outside of it. Additionally, they often require post‐hoc tools for interpretation. To avoid these limitations, we combine symbolic regression, sequential feature selection, and physical constraints in a hierarchical modeling framework. This framework allows us to discover new equations diagnosing cloud cover from coarse‐grained variables of global storm‐resolving model simulations. These analytical equations are interpretable by construction and easily transferable to other grids or climate models. Our best equation balances performance and complexity, achieving a performance comparable to that of NNs (R2 = 0.94) while remaining simple (with only 11 trainable parameters). It reproduces cloud cover distributions more accurately than the Xu‐Randall scheme across all cloud regimes (Hellinger distances < 0.09), and matches NNs in condensate‐rich regimes. When applied and fine‐tuned to the ERA5 reanalysis, the equation exhibits superior transferability to new data compared to all other optimal cloud cover schemes. Our findings demonstrate the effectiveness of symbolic regression in discovering interpretable, physically‐consistent, and nonlinear equations to parameterize cloud cover.

show abstract

“…One is the shape-constrained SR [14,15], which incorporates constraints on function shape (such as partial derivatives and monotonicity) using an efficient application of integer arithmetic. Additionally, other variations of SR direct the search for unit correctness [16][17][18], conserve physical properties [17,[19][20][21], and guide using predefined forms derived from the dataset [19,[22][23][24]. All of these approaches build on SR based on GAs, the original and most popular technique [1,13].…”

Section: Incorporating Background Knowledge Into Srmentioning

confidence: 99%

Incorporating background knowledge in symbolic regression using a computer algebra system

Fox,

Tran,

Nacion

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Symbolic Regression (SR) can generate interpretable, concise expressions that fit a given dataset, allowing for more human understanding of the structure than black-box approaches. The addition of background knowledge (in the form of symbolic mathematical constraints) allows for the generation of expressions that are meaningful with respect to theory while also being consistent with data. We specifically examine the addition of constraints to traditional genetic algorithm (GA) based SR (PySR) as well as a Markov-chain Monte Carlo (MCMC) based Bayesian SR architecture (Bayesian Machine Scientist), and apply these to rediscovering adsorption equations from experimental, historical datasets.We find that, while hard constraints prevent GA and MCMC SR from searching, soft constraints can lead to improved performance both in terms of search effectiveness and model meaningfulness, with computational costs increasing by about an order of magnitude. If the constraints do not correlate well with the dataset or expected models, they can hinder the search of expressions. We find incorporating these constraints in Bayesian SR (as the Bayesian prior) is better than by modifying the fitness function in the GA.

show abstract

Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws

Cited by 5 publications

References 58 publications

Data-Driven Equation Discovery of a Cloud Cover Parameterization

Data-Driven Equation Discovery of a Cloud Cover Parameterization

Data‐Driven Equation Discovery of a Cloud Cover Parameterization

Incorporating background knowledge in symbolic regression using a computer algebra system

Contact Info

Product

Resources

About