Choosing function sets with better generalisation performance for symbolic regression models

Nicolau, Miguel; Agapitos, Alexandros

doi:10.1007/s10710-020-09391-4

Cited by 19 publications

(15 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over and above attempts to evaluate derivatives at singular points, even protected division has the property that as x becomes very small but |x| > 0 , the quotient 1/x can become very large leading to numerical instabilities in the values predicted by the GP tree. We believe this is the underpinning reason for the conclusions of Nicolau and Agapitos [22] -see Ni et al [21] for a longer discussion; such numerical instabilities also appear to have been evident in the results of Dick et al [7] after (protected) division was included in their trees.…”

Section: Numerical Optimization Of the Constantsmentioning

confidence: 82%

“…Finally on this point, there is evidence in the literature [22,23] that suggests the composition of the function set can affect both evolutionary search and generalization performance. Under standardization, regressors which may originally all have been non-negative can acquire negative values.…”

Section: Discussion and Future Workmentioning

confidence: 99%

“…This has the advantage of guaranteeing that the function composition implemented by any GP tree is analytic, and therefore differentiable up to at least second order. Aside from the advantages of the AQ cited in [21], Nicolau and Agapitos [22] have recently reported that the AQ operator provides superior generalization over a range of regression problems compared to trees using protected division. Hence, given a tree for which we require to optimize the constant values, we can evaluate the necessary partial derivatives for SLSQP by: i) automatically generating the derivative trees of our target tree, and ii) evaluating those derivative trees by normal recursive tree traversals of the derivative trees for some given specific input.…”

Section: Numerical Optimization Of the Constantsmentioning

confidence: 99%

See 2 more Smart Citations

Constant optimization and feature standardization in multiobjective genetic programming

Rockett

2021

Genet Program Evolvable Mach

View full text Add to dashboard Cite

This paper extends the numerical tuning of tree constants in genetic programming (GP) to the multiobjective domain. Using ten real-world benchmark regression datasets and employing Bayesian comparison procedures, we first consider the effects of feature standardization (without constant tuning) and conclude that standardization generally produces lower test errors, but, contrary to other recently published work, we find much less clear trend for tree sizes. In addition, we consider the effects of constant tuning – with and without feature standardization – and observe that (1) constant tuning invariably improves test error, and (2) usually decreases tree size. Combined with standardization, constant tuning produces the best test error results; tree sizes, however, are increased. We also examine the effects of applying constant tuning only once at the end a conventional GP run which turns out to be surprisingly promising. Finally, we consider the merits of using numerical procedures to tune tree constants and observe that for around half the datasets evolutionary search alone is superior whereas for the remaining half, parameter tuning is superior. We identify a number of open research questions that arise from this work.

show abstract

Section: Numerical Optimization Of the Constantsmentioning

confidence: 82%

Section: Discussion and Future Workmentioning

confidence: 99%

Section: Numerical Optimization Of the Constantsmentioning

confidence: 99%

See 1 more Smart Citation

Constant optimization and feature standardization in multiobjective genetic programming

Rockett

2021

Genet Program Evolvable Mach

View full text Add to dashboard Cite

show abstract

“…To limit the complexity of the resulting expressions, the set of functions is restricted to the four elementary mathematical operations (addition, subtraction, product, and division). One can refer to [43] for a detailed study on the effect of the choice of function sets on the generalization performance of SR models.…”

Section: Methodsmentioning

confidence: 99%

A Genetic Programming Approach for Economic Forecasting with Survey Expectations

2022

View full text Add to dashboard Cite

We apply a soft computing method to generate country-specific economic sentiment indicators that provide estimates of year-on-year GDP growth rates for 19 European economies. First, genetic programming is used to evolve business and consumer economic expectations to derive sentiment indicators for each country. To assess the performance of the proposed indicators, we first design a nowcasting experiment in which we recursively generate estimates of GDP at the end of each quarter, using the latest business and consumer survey data available. Second, we design a forecasting exercise in which we iteratively re-compute the sentiment indicators in each out-of-sample period. When evaluating the accuracy of the predictions obtained for different forecast horizons, we find that the evolved sentiment indicators outperform the time-series models used as a benchmark. These results show the potential of the proposed approach for prediction purposes.

show abstract

“…Choosing an appropriate function set is a key decision in applying GP as it can have a vital impact on the performance of GP [ 4 , 5 ]. However, there is insufficient guidance in selecting a function set, and no systematic approach exists [ 6 ]. To date, it is also largely considered a decision made by domain experts.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Grammar Pruning for Program Size Reduction in Symbolic Regression

et al. 2023

View full text Add to dashboard Cite

Grammar is a key input in grammar-based genetic programming. Grammar design not only influences performance, but also program size. However, grammar design and the choice of productions often require expert input as no automatic approach exists. This research work discusses our approach to automatically reduce a bloated grammar. By utilizing a simple Production Ranking mechanism, we identify productions which are less useful and dynamically prune those to channel evolutionary search towards better (smaller) solutions. Our objective in this work was program size reduction without compromising generalization performance. We tested our approach on 13 standard symbolic regression datasets with Grammatical Evolution. Using a grammar embodying a well-defined function set as a baseline, we compare effective genome length and test performance with our approach. Dynamic grammar pruning achieved significantly better genome lengths for all datasets, while significantly improving generalization performance on three datasets, although it worsened in five datasets. When we utilized linear scaling during the production ranking stages (the first 20 generations) the results dramatically improved. Not only were the programs smaller in all datasets, but generalization scores were also significantly better than the baseline in 6 out of 13 datasets, and comparable in the rest. When the baseline was also linearly scaled as well, the program size was still smaller with the Production Ranking approach, while generalization scores dropped in only three datasets without any significant compromise in the rest.

show abstract

Choosing function sets with better generalisation performance for symbolic regression models

Cited by 19 publications

References 64 publications

Constant optimization and feature standardization in multiobjective genetic programming

Constant optimization and feature standardization in multiobjective genetic programming

A Genetic Programming Approach for Economic Forecasting with Survey Expectations

Dynamic Grammar Pruning for Program Size Reduction in Symbolic Regression

Contact Info

Product

Resources

About