Applications of probability density estimation to the detection of abnormal conditions in engineering

Abstract

A method is described for the identification of abnormal or unexpected conditions from measured response data. Such a technique would be useful in a wide range of engineering situations where a clear, early warning of an abnormal condition is required, but where classification of the specific abnormality is only of secondary importance. In this work, occurrences of unexpected operating conditions are indicated by measured data which exhibit a high degree of novelty with respect to that corresponding to normal conditions or responses. The proposed approach is based upon the probability density function (PDF ) estimation using a kernel method, the basis of which is described. The need for data compression in practical applications of PDF estimation is highlighted and a method demonstrated which is based on the wavelet transform. The combined data compression and PDF estimation approach for novelty detection is applied to data measured from a gearbox with a progressive fault and to radar data corresponding to six military targets. In both cases, abnormal situations are clearly identified on the basis of novel data inputs.

Anomaly detection on cup anemometers

“…This can be considered an example of the anomaly detection problem. In the industry, anomaly detection is usually addressed through three different approaches: case-based reasoning, model-based diagnosis and non-parametric models [14]. Regarding loss of performance in cup anemometers, the results of the PHM 2011 Data Challenge Competition are particularly interesting.…”

Study of Self-Similarity for Detection of Rate-based Network Anomalies

“…The only difference is in the way density estimation technique is used. Desforges et al [60], proposed a semi-supervised statistical technique to detect anomalies, which uses kernel functions to estimate the probability distribution function (pdf) for the normal instances. A new instance, which lies in the low probability area of this pdf is declared to be anomalous.…”

Adaptive Mixture Discriminant Analysis for Supervised Learning with Unobserved Classes

“…Neural networks and kernel methods have been also widely used for novelty detection. Bishop [7] used parametric statistics by post-processing neural networks for detecting new data distribution whereas a probability density estimation of neural network outputs is used in [16] as a measure of novelty. Another approach based on neural networks was proposed in [36] which used a thresholding on the neural network output for detection new samples.…”