Point-Wise Confidence Interval Estimation by Neural Networks: A Comparative Study based on Automotive Engine Calibration

Abstract

In developing neural network techniques for real world applications it is still very rare to see estimates of con dence placed on the neural network predictions. This is a major de ciency, especially in safety-critical systems. In this paper we explore three distinct methods of producing point-wise con dence intervals using neural networks. We compare and contrast Bayesian, Gaussian Process and Predictive error bars evaluated on real data. The problem domain is concerned with the calibration of a real automotive engine management system for both air-fuel ratio determination and on-line ignition timing. This problem requires real-time control and is a good candidate for exploring the use of con dence predictions due to its safety-critical nature.

Using Principal Component Scores and Artificial Neural Networks in Predicting Water Quality Index

“…If the residual errors are randomly distributed, there is a general rule of thumb which states that they will lie within two standard deviations of their mean with a probability of 0.95. This method was used in the measurements of the ANN prediction performance conducted by some researchers (Bishop, 1995;Tibshirani, 1996;Shao et al, 1997;Zhang et al, 1998;Lowe and Zapart, 1999;Townsend and Tarassenko, 1999) ANN models and statistical analyses were carried out using MATLAB 7.0 and XLSTAT2008 (Excel2003 add-in) for Windows. …”

Robust control of nonlinear stochastic systems by modelling conditional distributions of control signals

“…If this variance is used as a target value for another neural network, then the optimum output of this second network is again the conditional mean of that variance. As reported in Lowe and Zapart [5], in the implementation of predictive error bars two correlated neural neural networks are used. Each network shares the same input and hidden nodes, but has different final layer links which are estimated to give the approximated conditional mean of the target data in the first network, and the approximated conditional mean of the variance in the second network.…”

“…Different methods for estimating the uncertainty around the predicted output of a neural network have been presented [4]- [6]. In this work, the predictive error bar method will be used [5]. This approach is based on the important result that for a network trained on minimum square error, the optimum network output approximates the conditional mean of the target data, or f À1 opt ðsðtÞÞ ¼< uðt À dÞjsðtÞ >, and that the local variance of the target data can be estimated as kuðt À dÞ À f À1 opt ðsðtÞÞk 2 .…”

A Multi-Net System for the Fault Diagnosis of a Diesel Engine

“…Although there are several examples of the application of ANNs to problems of fault diagnosis, there are fewer examples of their application to fault diagnosis in internal combustion engines (although for exceptions, see elsewhere [1,[10][11][12]). More common applications have been to gear boxes, and bearing faults (e.g.…”