Finding an Unknown Number of Multivariate Outliers

Abstract

We use the forward search to provide parameter estimates for Mahalanobis distances used to detect the presence of outliers in a sample of multivariate normal data. Theoretical results on order statistics and on estimation in truncated samples provide the distribution of our test statistic. Comparisons of our procedure with tests using other robust Mahalanobis distances show the good size and high power of our procedure. We also provide a unification of results on correction factors for estimation from truncated samples.

Assessing trimming methodologies for clustering linear regression data

“…This means that the robust distances are compared with the confidence bands at a selected confidence level, and the observations with distances exceeding the bands are trimmed. In this case the multivariate outlier detection procedure proposed by Cerioli (2010), based on the reweighted MCD estimator (Rousseeuw and Van Driessen 1999), or the Forward Search (Riani et al 2009) can be used at each concentration step of each starting subset. The observations surviving to the two trimming steps are then used for updating the regression coefficients, weights and scatter matrices.…”

“…5 Boxplots of the classification errors (left panels) and group-wise slope biases (right panels) obtained in the 1000 data configurations corresponding to lines 5 (top panels) and 6 (bottom panels) of Table 1 calculated at a confidence level specified by the user. The identification of the units is done using either the Finite Sample Re-weighted MCD rule (Cerioli 2010) or the Forward Search (Riani et al 2009) for their good trade-off between robustness and efficiency. TCLUST (García-Escudero et al 2008), which in the univariate case is equivalent to the MCD, can be also used.…”

“…To address the problem we have introduced a new methodological option in TCLUST-REG, through the possibility to regulate the percentage of second level trimming during the estimation steps. This is done through an adaptive approach based either on the Forward Search (Riani et al 2009), or on the Finite Sample Re-weighted MCD method of Cerioli (2010).…”

Robust Clustering for Performance Evaluation

“…Examples of such envelopes and their use in the forward search for cluster ing moderate sized data sets are presented by Atkinson et al (2006) and , in which the largest example has 1,000 observations. The theoret ical results of Riani et al (2009) provide the tools for extending our methodology to larger data sets, where indeed inspection of the trajectory of a single minimum Mahalanobis distance, defined in (3), greatly simplifies the cluster identification process. In Bini et al (2004) we applied earlier versions of these methods to the analysis of a complicated set of data on the performance of Italian universities.…”

Asymptotic Analysis of the Forward Search

“…This approach has been suggested, analysed and applied for regression models in the monograph Atkinson and Riani (2000), see also Atkinson, Riani and Cerioli (2010a) for a recent overview, while R and matlab code is freely available from www.riani.it. Riani, Atkinson and Cerioli (2009) discuss the application of the Forward Search to multivariate location-scale models. So far formal asymptotic analysis has not been undertaken and inferential procedures are relying on a calibrated distribution approximation, see Riani and Atkinson (2007).…”