Deterministic symbolic regression with derivative information: General methodology and application to equations of state

Abstract

Symbolic regression methods simultaneously determine the model functional form and the regression parameter values by generating expression trees. Symbolic regression can capture the complexity of real‐world phenomena but the use of deterministic optimization for symbolic regression has been limited due to the complexity of the search space of existing formulations. We present a novel deterministic mixed‐integer nonlinear programming formulation for symbolic regression that incorporates derivative constraints through auxiliary expression trees. By applying the chain rule to mathematical operations, binary expression trees are capable of representing the calculation of first and second derivatives. We apply this formulation to illustrative examples using derivative information to show increased model discrimination capability. In addition, we perform a case study of a thermodynamic equation of state to gain insight on valid functional forms with thermodynamics‐based constraints on the first and second derivatives.

Incorporating background knowledge in symbolic regression using a computer algebra system

“…All of these approaches build on SR based on GAs, the original and most popular technique [1,13]. In contrast, Engle and Sahinidis developed a deterministic SR algorithm (using mixed-integer nonlinear programming) that constrains the equation space to functions that obey derivative constraints from theory, improving the quality of expressions for thermodynamic equations of state [25]. Another approach is the Bayesian machine scientist (BMS) [26].…”

Artificial Intelligence in Physical Sciences: Symbolic Regression Trends and Perspectives

“…The latter, although extremely fast, produces non-interpretable equations [ 84 ]. In addition, there also exist Mixed-Integer Non-Linear Programming (MINLP) formulations [ 85 , 86 ], models that identify the SR problem as a linguistic [ 87 ], others that incorporate probabilistic features such as probabilistic framework [ 88 ] or probabilistic grammars [ 89 ], Bayesian approaches [ 90 ] and more [ 91 , 92 ].…”

Evolving scientific discovery by unifying data and background knowledge with AI Hilbert

“…By leveraging fundamental results from real algebraic geometry, we obtain formal proofs of the correctness of our laws as a byproduct of the optimization problems. This is notable, because existing automated approaches to scientific discovery 22 – 25 , as reviewed in Section 1 of our supplementary material , often rely upon deep learning techniques that do not provide formal proofs and are prone to hallucinating incorrect scientific laws that cannot be automatically proven or disproven, analogously to output from state-of-the-art Large Language Models such as GPT-4 26 . As such, any new laws derived from these systems cannot easily be explained or justified.…”