Confidence in data mining model predictions: a financial engineering application

Abstract

Absrracf-This paper describes a generally applicable robust method for determining prediction intervals for models derived by non-linear regression. Hypothesis tests for bias are applied. The concept is demonstrated by application to a standard synthetic example, and is then applied to prediction intervals for a financial engineering example viz. option pricing using data from LlFFE for 'ESX' European style options on the FTSE 100 index. Unbiased estimates of the standard error are obtained. The method uses standard regression procedures to determine local error bars and avoids programming special architectures. It is appropriate for target data with non-constant variance.

Data mining to support anaerobic WWTP monitoring

“…For regression involving neural nets (NN) then sensitivity analysis can be used to rank the sensors. Useful measures of model accuracy can be derived for the out-of-sample test data set from the summary statistics such as mean error, mean-squared error, prediction risk, and R 2 ; however, prediction intervals can provide a guide on a point basis to the quality of the prediction across the sensor range and so will be given here also (Healy, Dixon, Read, & Cai, 2003a).…”

“…A robust method for estimating local error bars has been developed by Healy, Dixon, Read, and Cai (2003a, b) and shown to be effective for a range of synthetic and multivariate data. The method's main theoretical advantage was that it maintained the integrity of statistical inference.…”

Confidence limits for data mining models of options prices

“…In this paper we describe a robust method for determining prediction intervals for neural nets and related techniques. We have tested the method empirically using a standard synthetic data set, and compared it with a method restricted to neural nets [6]. Here, our method is applied to obtain prediction intervals for pricing options, using a data set of 14,257 LIFFE European style FTSE 100 index 'ESX' options [7].…”

“…Standard methods of computing confidence intervals such as the 'delta', and 'sandwich' methods as well as the 'naive' bootstrap apply in principle but have practical problems as discussed in Refs. [6,3]. Nix and Weigend [8] describe a method of using a neural net to estimate the variance of its own predictions of the target variable, thereby allowing prediction intervals to be constructed.…”

Constructing prediction intervals for neural network metamodels of complex systems

“…It is expected that the target will lie within the interval with a prescribed probability that is called the confidence level in literature. From a practical standpoint, prediction intervals for unseen data are of more importance than confidence intervals of the true regression [18] [20] and are typically wider [21].…”