Outlier Detection in Multivariate Analytical Chemical Data

Abstract

The unreliability of multivariate outlier detection techniques such as Mahalanobis distance and hat matrix leverage has been known in the statistical community for well over a decade. However, only within the past few years has a serious effort been made to introduce robust methods for the detection of multivariate outliers into the chemical literature. Techniques such as the minimum volume ellipsoid (MVE), multivariate trimming (MVT), and M-estimators (e.g., PROP), and others similar to them, such as the minimum covariance determinant (MCD), rely upon algorithms that are difficult to program and may require significant processing times. While MCD and MVE have been shown to be statistically sound, we found MVT unreliable due to the method's use of the Mahalanobis distance measure in its initial step. We examined the performance of MCD and MVT on selected data sets and in simulations and compared the results with two methods of our own devising. Both the proposed resampling by the half-means method and the smallest half-volume method are simple to use, are conceptually clear, and provide results superior to MVT and the current best-performing technique, MCD. Either proposed method is recommended for the detection of multiple outliers in multivariate data.

Quantitative mass spectrometry in proteomics: a critical review

“…During calculation of protein values from individual spectra by linear regression (see above) outlier detection on the spectrum level is possible using established methods [113,114] but may result in loss of valuable data. For data correction at the protein level, methods for multivariate data can also be adapted [115,116].…”

Critical Aspects of Supervised Pattern Recognition Methods for Interpreting Compositional Data

“…Once outliers have been deleted, researchers usually remove them from the data set, but outliers could be corrected before applying the definite mathematical procedures by using robust algorithms . Robust methods give better results, specially some improved algorithms such as resampling by the half-means (RHM) and smallest half-volume (SHV) (Egan & Morgan , 1998).…”

In-situ calibration of pulsed eddy current detection of cracks at fasteners in CP-140 aircraft

“…In statistical terms, this means distinguishing between members of a cluster and outliers. This is a common challenge and an entire branch of statistics, known as robust statistics is being developed to address this [5][6][7][8][9][10][11][12]. In Robust statistics a fraction of the data, h, which is most tightly clustered is used to estimate the statistical properties of the uncontaminated population.…”