Fast Estimation of Covariance Parameters in Least-Squares Collocation by Fisher Scoring with Levenberg–Marquardt Optimization

Abstract

Maximum likelihood (ML) and restricted maximum likelihood (REML) are nowadays very popular in geophysics, geodesy and many other fields. There is also a growing number of investigations into how to calculate covariance parameters by ML/ REML accurately and fast, and assure the convergence of the iteration steps in derivativebased approaches. The latter condition is not satisfied in many solutions, as it requires composed procedures or takes an unacceptable amount of time. The article implements efficient Fisher scoring (FS) to covariance parameter estimation in least-squares collocation (LSC). FS is optimized through Levenberg-Marquardt (LM) optimization, which provides stability in convergence when estimating two covariance parameters necessary for LSC. The motivation for this work was a very large number of non-optimized FS in the literature, as well as a deficiency of its scientific and engineering applications. The example work adds some usefulness to maximum likelihood estimation (ML) and FS and shows a new application-an alternative approach to LSC-a parametrization with no empirical covariance estimation. The results of LM damping applied to FS (FSLM) require some additional research related with optimal LM parameter. However, the method appears to be a milestone in relation to non-optimized FS, in terms of convergence. The FS with LM provides a reliable convergence, whose speed can be adjusted by manipulating the LM parameter.

Geodesy and Cartography

“…It allows for fitting of geodetic network to unstable reference points (Osada et al, 2017a(Osada et al, , 2018. -Development of the covariance function parametrisation approach that is based on the Fisher scoring technique and the Levenberg-Marquardt optimisation (Jarmołowski, 2015(Jarmołowski, , 2017. -Development of various strategies of network adjustment in the presence of gross errors in observation sets and instabilities of reference system (Osada et al, 2017a(Osada et al, , 2018Zienkiewicz, 2015;Nowel, 2015Nowel, , 2016aNowel, , 2016b.…”

“…Many observations, with errors several times larger than the average error value, have been found in the dataset. The fast estimation of covariance parameters in least squares collocation by means of the Fisher scoring and the Levenberg-Marquardt optimisation has been further investigated by Jarmołowski (2017). The author performed several numerical tests and figured out that the Fisher scoring technique optimised through Levenberg-Marquardt and applied in the parametrisation of the least squares collocation is much faster than any other technique.…”

On the drawback of local detrending in universal kriging in conditions of heterogeneously spaced regional TEC data, low-order trends and outlier occurrences

“…The oldest and well-known method of the parameter selection is the fitting of analytical model into its corresponding empirical representation (Posa 1989;Barzaghi et al 2003;Samui and Sitharam 2011). This method is quite effective for the signal parameters; however, supplementary investigations have been undertaken in order to determine noise parameters more accurately (Marchenko et al 2003;Peng and Wu 2014;Jarmołowski 2016Jarmołowski , 2017. The estimates of signal and noise covariance parameters are very often achieved by different cross-validation techniques, e.g., hold-out (HO) validation (Arlot and Celisse 2010;Kohavi 1995) or leave-one-out (LOO) validation (Kohavi 1995;Behnabian et al 2018).…”

“…The estimates of signal and noise covariance parameters are very often achieved by different cross-validation techniques, e.g., hold-out (HO) validation (Arlot and Celisse 2010;Kohavi 1995) or leave-one-out (LOO) validation (Kohavi 1995;Behnabian et al 2018). Another solution that can be applied in the parametrization procedures is maximum likelihood estimation (MLE) of parameters for kriging (Pardo-Igúzquiza et al 2009;Todini 2001;Zimmermann 2010) or for LSC (Jarmołowski 2015(Jarmołowski , 2017.…”