An attempt to analyse Iterative Data Snooping and L1-norm based on Monte Carlo simulation in the context of leveling networks

Abstract

The goal of this paper is to evaluate the outlier identification performance of iterative Data Snooping (IDS) and L1-norm in leveling networks by considering the redundancy of the network, number and size of the outliers. For this purpose, several Monte-Carlo experiments were conducted into three different leveling networks configurations. In addition, a new way to compare the results of IDS based on Least Squares (LS) residuals and robust estimators such as the L1-norm has also been developed and presented. Two different scenarios were considered in that comparison: (i) both IDS and L1-norm evaluated with the same threshold values; (ii) both IDS and L1-norm compared with the same false positive rates. In latter case, a Monte-Carlo approach was applied to control the false positive rates. The question of which of them performs better depends on the viewpoint.From the perspective of analysis only according to the success rate, it is shown that L1-norm performs better than IDS for the case of networks with low redundancy (𝑟̅ < 0.5), especially for cases where more than one outlier is present in the dataset. In the relationship between false positive rate and outlier identification success rate, however, IDS performs better than L1-norm. In that case, IDS with a critical value of 3.29 has the best cost-benefit ratio, independently of the levelling network configuration, number and size of outliers.

Robust Estimation of Deformation from Observation Differences Using Some Evolutionary Optimisation Algorithms

“…The test sample was extended with one characteristic displacement scenario where all network points (PRPs and object points) are undisplaced ) in order to examine the false-positive rate (FPR) of the proposed GREDOD modification, as reported in [ 59 ], where a similar procedure was conducted for outlier analysis. FPR is defined as the number of false-positive rates divided by the total number of experiments.…”

Monte Carlo-Based Covariance Matrix of Residuals and Critical Values in Minimum L1-Norm

“…Recently, many works (see, e.g., [9,15,16]) have investigated critical values for normalized residuals in IDS based on MCS control of false positive rate. Motivated by mentioned works, we propose the following procedure for any robust estimator (Figure 2).…”

“…In this context, Lehmann [9] showed that the critical values cannot be derived from well-known univariate test distributions (e.g., normal distribution), due to the correlations between residuals. Hence, in order to fully control false positive rates in geodetic networks, critical values for IDS started to be numerically computed by means of Monte Carlo simulation (MCS) (see, e.g., [15,16]).…”

“…where p is the weight vector of uncorrelated observations and |v| is the vector of absolute residuals. In particular, Minimum L1-norm is likely to provide higher outlier identification success rate than IDS for low-redundancy networks [15,22]. e minimum L1-norm solution may not be unique [23], and its vector of residuals in geodetic networks tends to be sparse, with many residuals being equal to zero (see, e.g., [20,24]).…”

“…(1) the ratio between residuals and respective observation standard deviation σ i as the test statistic (see, e.g., [15,41]); and (2) the normalized residual w i (Equation 2, considering v i and σ v i from minimum L1-norm) as the test statistic (see, e.g., [21,42]). For both criteria, the chosen critical values (from which the respective observation would be classified as an outlier) were, in general, common values taken from the univariate normal statistical table, such as |Z α/2 | � 3.00 (α � 0.27%) and |Z α/2 | � 3.29 (α � 0.1%), which, as mentioned, are not appropriate even for IDS.…”