A Preliminary Investigation of Overfitting in Evolutionary Driven Model Induction: Implications for Financial Modelling

Abstract

Abstract. This paper investigates the effects of early stopping as a method to counteract overfitting in evolutionary data modelling using Genetic Programming. Early stopping has been proposed as a method to avoid model overtraining, which has been shown to lead to a significant degradation of out-of-sample performance. If we assume some sort of performance metric maximisation, the most widely used early training stopping criterion is the moment within the learning process that an unbiased estimate of the performance of the model begins to decrease after a strictly monotonic increase through the earlier learning iterations. We are conducting an initial investigation on the effects of early stopping in the performance of Genetic Programming in symbolic regression and financial modelling. Empirical results suggest that early stopping using the above criterion increases the extrapolation abilities of symbolic regression models, but is by no means the optimal training-stopping criterion in the case of a real-world financial dataset.

Evolving Interpolating Models of Net Ecosystem CO2 Exchange Using Grammatical Evolution

“…It can be seen that the model does not overfit the data. Comparison with runs using all the available data for training achieved similar results, suggesting that the use of a 2-set methodology neither hinders nor improves the performance of the obtained models, confirming previous results reported in GP [5] and GE [23]. The best (D1 + N2) model is shown in Eq.…”

Tackling Overfitting in Evolutionary-Driven Financial Model Induction

“…In Fig. 8.3(b) we can see an explosion in the test error towards the end of the run [23], which contrasts with the low value of the training error.…”

“…Table 8.1 shows results of interest with respect to the fitness as evaluated on the validation and test dataset, for 9 runs. It shows that stopping evolution before the specified number of generations had elapsed, in the majority of cases would have led to the model extrapolating better beyond the range in which it was trained [23]. Early stopping has been described in Section 8.2.2.…”

“…We examine the eight runs where the model that we would have evolved using traditional early stopping had better test fitness than the model that would have been evolved at the end of the run. In five out of these eight runs, the optimal generation at which to stop (as measured by test fitness) came later than the generation of the result of training using traditional early stopping [23].…”

Early stopping criteria to counteract overfitting in genetic programming

“…The question then becomes, when should early stopping take place? Previous work [9] indicates that early stopping should not necessarily take place the first time validation set error disimproves during a symbolic regression run using Grammarbased GP. With the aim of developing techniques to counteract overfitting in Grammar-based GP, the classes of stopping criteria in [8] were implemented here on symbolic regression problems.…”