Experimental quality evaluation of lattice basis reduction methods for decorrelating low-dimensional integer least squares problems

Abstract

Reduction can be important to aid quickly attaining the integer least squares (ILS) estimate from noisy data. We present an improved LLL algorithm with fixed complexity by extending a parallel reduction method for positive definite quadratic forms to lattice vectors. We propose the minimum angle of a reduced basis as an alternative quality measure of orthogonality, which is intuitively more appealing to measure the extent of orthogonality of a reduced basis. Although the LLL algorithm and its variants have been widely used in practice, experimental simulations were only carried out recently and limited to the quality measures of the Hermite factor, practical running behaviors and reduced Gram-Schmidt coefficients. We conduct a large scale of experiments to comprehensively evaluate and compare five reduction methods for decorrelating ILS problems, including the LLL algorithm, its variant with deep insertions and our improved LLL algorithm with fixed complexity, based on six quality measures of reduction. We use the results of the experiments to investigate the mean running behaviors of the LLL algorithm and its variants with deep insertions and the sorted QR ordering, respectively. The improved LLL algorithm with fixed complexity is shown to perform as well as the LLL algorithm with deep insertions with respect to the quality measures on length reduction but significantly better than this LLL variant with respect to the other quality measures. In particular, our algorithm is of fixed complexity, but the LLL algorithm with deep insertions could seemingly not be terminated in polynomial time of the dimension of an ILS problem. It is shown to perform much better than the other three reduction methods with respect to all the six quality measures. More than six millions of the reduced Gram-Schmidt coefficients from each of the five reduction methods clearly show that they are not uniformly distributed but depend on the reduction algorithms used. The simulation results of the reduced Gram-Schmidt coefficients have clearly shown that our improved LLL algorithm tends to produce small reduced Gram-Schmidt coefficients near zero with a larger probability and large reduced Gram-Schmidt coefficients near both ends of 0.5 and −0.5 with a smaller probability.

Resolve integer ambiguity based on the global deep grid-based algorithms

“…The goal of lattice basis reduction is to transform the basis vectors of a lattice to make them shorter and more orthogonal, thus simplifying computations and improving efficiency 20 . In order to measure the performance of lattice basis reduction, the Hadamard ratio is often used as an evaluation index 21 . The Hadamard ratio is used to describe the degree of orthogonality of a set of vectors, with a range of (0, 1] .…”

High-rate multi-GNSS attitude determination: experiments, comparisons with inertial measurement units and applications of GNSS rotational seismology to the 2011 Tohoku Mw9.0 earthquake

“…In principle, the integer LS method can be directly applied to (1) to resolve the integer parameters z, as first described for GNSS precise relative positioning in geodesy by Teunissen (1995) and further significantly improved by Chang et al (2005) and Xu et al (2012). For more mathematical reports on integer LS and reduction, the reader is referred to Lenstra et al (1982), Fincke andPohst (1985), Schnorr and Euchner (1994) and Xu (2001Xu ( , 2006Xu ( , 2012Xu ( , 2013. Nevertheless, since the multi-antenna configuration can be measured precisely a priori, a number of baseline-constrained ambiguity resolution methods have been developed under the framework of GNSS attitude determination (Lu 1995, Wang et al 2009, Teunissen 2012a.…”

Improved GNSS Ambiguity Fast Estimation Reduction Algorithm

“…In measuring the performance of lattice basis reduction, an orthogonal defect (OD) is usually used to reflect the orthogonality of the basis vector, but it has an obvious disadvantage in that only the OD value is obtained, which is not able to intuitively judge the extent of the orthogonality of the reduced basis [37][38][39]. Therefore, in this paper, the minimum angle θ of the reduced basis vector is used instead of the orthogonality defect to measure the extent of the orthogonality of the reduced basis.…”