Evaluating prognostics performance for algorithms incorporating uncertainty estimates

Abstract

Uncertainty Representation and Management (URM) are an integral part of the prognostic system development. 1 As capabilities of prediction algorithms evolve, research in developing newer and more competent methods for URM is gaining momentum. 2 Beyond initial concepts, more sophisticated prediction distributions are obtained that are not limited to assumptions of Normality and unimodal characteristics. Most prediction algorithms yield non-parametric distributions that are then approximated as known ones for analytical simplicity, especially for performance assessment methods. Although applying the prognostic metrics introduced earlier with their simple definitions has proven useful, a lot of information about the distributions gets thrown away. In this paper, several techniques have been suggested for incorporating information available from Remaining Useful Life (RUL) distributions, while applying the prognostic performance metrics. These approaches offer a convenient and intuitive visualization of algorithm performance with respect to metrics like prediction horizon and α-λ performance, and also quantify the corresponding performance while incorporating the uncertainty information. A variety of options have been shortlisted that could be employed depending on whether the distributions can be approximated to some known form or cannot be parameterized. This paper presents a qualitative analysis on how and when these techniques should be used along with a quantitative comparison on a real application scenario. A particle filter based prognostic framework has been chosen as the candidate algorithm on which to evaluate the performance metrics due to its unique advantages in uncertainty management and flexibility in accommodating non-linear models and non-Gaussian noise. We investigate how performance estimates get affected by choosing different options of integrating the uncertainty estimates. This allows us to identify the advantages and limitations of these 1 978-1-4244-3888-4

An ensemble of models for integrating dependent sources of information for the prognosis of the remaining useful life of Proton Exchange Membrane Fuel Cells

“…In order to evaluate the average performance of RUL predictions, the common way is to apply several RUL predictions at different time steps to obtain a sequence of predicted RULs [43,44].…”

Particle-Filtering-Based Discharge Time Prognosis for Lithium-Ion Batteries With a Statistical Characterization of Use Profiles

“…The PDF for the remaining useful life of the faulty system, or equivalently its Time-of-Failure, can then be computed by combining information from these long-term predictions and system hazard zones (areas of the state space that are associated with critical conditions for the system) [33], [34]. Focusing on the analysis of the EOD [18], the hazard zone may be characterized as a threshold for the SOC.…”

Physics-based prognostic modelling of filter clogging phenomena

“…A research group from NASA have been conducting a comprehensive research on the standardisation of prognostic evaluation metrics [45], [46]. [47], [48] and [49]. They have introduced a hierarchical group of prognostic evaluation metrics.…”