A Monte Carlo-Based Outlier Diagnosis Method for Sensitivity Analysis

Abstract

An iterative outlier elimination procedure based on hypothesis testing, commonly known as Iterative Data Snooping (IDS) among geodesists, is often used for the quality control of modern measurement systems in geodesy and surveying. The test statistic associated with IDS is the extreme normalised least-squares residual. It is well-known in the literature that critical values (quantile values) of such a test statistic cannot be derived from well-known test distributions but must be computed numerically by means of Monte Carlo. This paper provides the first results on the Monte Carlo-based critical value inserted into different scenarios of correlation between outlier statistics. From the Monte Carlo evaluation, we compute the probabilities of correct identification, missed detection, wrong exclusion, over-identifications and statistical overlap associated with IDS in the presence of a single outlier. On the basis of such probability levels, we obtain the Minimal Detectable Bias (MDB) and Minimal Identifiable Bias (MIB) for cases in which IDS is in play. The MDB and MIB are sensitivity indicators for outlier detection and identification, respectively. The results show that there are circumstances in which the larger the Type I decision error (smaller critical value), the higher the rates of outlier detection but the lower the rates of outlier identification. In such a case, the larger the Type I Error, the larger the ratio between the MIB and MDB. We also highlight that an outlier becomes identifiable when the contributions of the measures to the wrong exclusion rate decline simultaneously. In this case, we verify that the effect of the correlation between outlier statistics on the wrong exclusion rate becomes insignificant for a certain outlier magnitude, which increases the probability of identification.

Which Are the Effects of Hard and Soft Equality Constraints on the Iterative Outlier Elimination Procedure?

“…Failure to identify an outlier can jeopardise the reliability level of a system. Due to its importance, outliers must be appropriately treated to ensure the quality of data analysis [45].…”

“…IDS is an iterative outlier elimination procedure, which combines estimation, testing and a corrective action [45,48]. Parameter estimation is often conducted in the sense of the LS.…”

“…There are chances of correct and false decisions of IDS, because these procedure is based on statistical hypothesis testing. Recently, Rofatto et al [45] have provided an algorithm based on Monte Carlo to determine the probability levels associated with IDS. In that case, they described six classes of decisions for IDS, namely probability of correct identification (P CI ), probability of missed detection (P MD ), probability of wrong exclusion (P WE ), probability of over-identification positive (P over+ ), probability of over-identification negative (P over− ) and probability of statistical overlap (P ol ), as follows:…”

“…The localization of the outlier is a problem of "outlier identification", i.e. "outlier identification" implies the execution of a search among the measurements for the most likely outlier [45]. Therefore, the smallest value of an outlier that can be detected given a certain P CD defines the MDB.…”

“…We show that the clusters can be defined according to two deterministic parameters: local redundancy and correlation between the outlier test statistics. Moreover, critical values optimized by Monte Carlo method were used here [45,46] in order to compute the decision classes associated with IDS, i.e. P CI , P MD , P WE , P over+ , P over− and P ol .…”