Shape-Constrained Symbolic Regression—Improving Extrapolation with Prior Knowledge

Abstract

We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce e.g. monotonicity of the function over selected inputs. The aim is to find models which conform to expected behaviour and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shapeconstrained symbolic regression: i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and ii) a two population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.

A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge

“…For example, 58 suggested that physical models guarantee monotonic behavior concerning some of its features and narrow the search space to include only monotonic functions. Extending this line of thought, 59 , 60 suggested adding knowledge about convexity instead of only looking at monotonicity. Unlike monotonicity, the addition of convexity constraint does not stem from physical reasoning but rather from the necessity to formulate physical models as functions that can be optimized efficiently.…”

Shape-constrained Symbolic Regression with NSGA-III

“…The most related article for this work is presented in [8]. The authors introduce SCSR a method, which allows to use prior knowledge in a data-based modelling approach.…”

“…In [8] the authors proposed shape-constrained symbolic regression (SCSR) a supervised machine learning approach that aims at both fitting trainings data and compliance with the given shape-constraints. It is shown that the resulting models, fit the training data well and even achieve slightly better training errors than standard genetic programming (GP) models in some cases, and that the models conform to the specified constraints.…”

Multi-Objective Optimization with Transformation-Interaction-Rational Representation for Symbolic Regression

“…The framework allows the user to choose whether to input the expected domain of each variable or to estimate from the input data. This library is also combined with the shape-constraint 2 library to allow a high-level description of Shape-constraints [4,5] The srtree library 3 is responsible for managing and evaluating the expression trees. It supports optimized evaluation of vectorized data and the calculation of the derivative of any order for a given expression.…”

“…These techniques have the advantage of having an extensive set of tools created to find an optimal parameters set. On the other hand, having a fixed form can limit the possible shapes the regression model can fit, limiting their extrapolation capabilities [4,5].…”