Assessment of inner reliability in the Gauss-Helmert model

Abstract

In this contribution, the minimum detectable bias (MDB) as well as the statistical tests to identify disturbed observations are introduced for the Gauss-Helmert model. Especially, if the observations are uncorrelated, these quantities will have the same structure as in the Gauss-Markov model, where the redundancy numbers play a key role. All the derivations are based on one-dimensional and additive observation errors respectively offsets which are modeled as additional parameters to be estimated. The formulas to compute these additional parameters with the corresponding variances are also derived in this contribution. The numerical examples of plane fitting and yaw computation show, that the MDB is also in the GHM an appropriate measure to analyze the ability of an implemented least-squares algorithm to detect if outliers are present. Two sources negatively influencing detectability are identified: columns close to the zero vector in the observation matrix B and sub-optimal configuration in the design matrix A. Even if these issues can be excluded, it can be difficult to identify the correct observation as being erroneous. Therefore, the correlation coefficients between two test values are derived and analyzed. Together with the MDB these correlation coefficients are an useful tool to assess the inner reliability – and therefore the detection and identification of outliers – in the Gauss-Helmert model.

On the making of Ptolemy’s star catalog

“…Further inconsistent data are identified by the iterative statistical method data snooping (see Ettlinger and Neuner 2020). In doing so, an adjustment is carried out in every iteration step, and based on its results the stochastic variables are each tested for gross errors by a statistical test (see Appendix C).…”

“…where r i is the redundancy number of the ith stochastic variable (cf. Ettlinger and Neuner 2020). If the accuracy of the stochastic variables is unknown, the externally studentized residual…”

The determination of Hipparchus' phenomena and their consistency with the Almagest star catalog

“…For the identification of further inconsistent data, the iterative statistical method data snooping (e.g., Ettlinger & Neuner 2020) is used. In every iteration step, an adjustment according to Section 3.1 is computed, and the stochastic variables are each tested for gross errors (outliers) by the externally studentized residual (e.g., Jäger et al 2005, p. 193), where v i is the residual of the i th stochastic variable, r i is its redundancy number (see e.g., Ettlinger & Neuner 2020), S is the weighted sum of squared residuals, and f is the number of degrees of freedom of the adjustment (for the present type of adjustment, f = No. of condition equations − 2; two‐sided test).…”

“…For the estimation of the unknown parameters, the method of least squares is applied. The numerical solution is achieved by a linearization of the condition equations and an iterative solving (e.g., Ettlinger & Neuner 2020). For the linearization, the analytical expressions of the partial derivatives of the condition equations with respect to the parameters and stochastic variables were determined by the application Mathematica (Wolfram Research, Inc. 2005).…”

On the Reliability of Mixed Ls Adjustment Models

“…Wang et al (2020) applied Baarda's data snooping algorithm for the equality constrained, nonlinear GHM while using sensitivity analysis. Some aspects of minimum detectable bias (MDB) and statistical tests to identify outliers for the GHM are presented in Ettlinger and Neuner (2020).…”