Greedy Motzkin–Kaczmarz methods for solving linear systems

Abstract

The famous greedy randomized Kaczmarz (GRK) method uses the greedy selection rule on maximum distance to determine a subset of the indices of working rows. In this paper, with the greedy selection rule on maximum residual, we propose the greedy randomized Motzkin-Kaczmarz (GRMK) method for linear systems. The block version of the new method is also presented. We analyze the convergence of the two methods and provide the corresponding convergence factors. The computational complexities of the proposed methods are also given.Extensive numerical experiments show that the GRMK method has almost the same performance as the GRK method for dense matrices and the former performs better in computing time for some sparse matrices and some realistic practical problems, and their block versions have almost the same performance for most of cases.

Regularized randomized iterative algorithms for factorized linear systems

“…Our numerical examples indicate that the proposed algorithms can find sparse (least squares) solutions and can be faster than the RSK and ExSRK algorithms for the corresponding full linear system. Acceleration strategies such as those used in [15,1,2,14,22,9] can be integrated into our algorithms easily and the corresponding convergence analysis is straightforward. The extension to a factorized linear system with rank-deficient A and B will be the future work.…”

On the Polyak momentum variants of the greedy deterministic single and multiple row‐action methods

“…Its flexibility allows us to tune the relaxation parameters and has led to several popular approaches. Specifically, the maximal weighted residual rule 16,31 emerges if one takes $$$ \theta =1 $$$, which is a generalized version of Motzkin's rule 32 by multiplying the norm of the corresponding row of the coefficient matrix; see also References 33 and 34. Later, the block version of (4) was given by Miao and Wu 14 .…”

“…Very recently, a probability distribution depending on the angle was used to determine the row index 38 . We refer to References 33,34,39–41 and the references therein for additional details on greedy row selection.…”

Splitting-based randomized iterative methods for solving indefinite least squares problem

“…Later, this convergence rate was further accelerated by using various strategies including block strategies (see, e.g., Needell & Tropp, 2014;Necoara, 2019;Du et al, 2020;Zhang & Li, 2021), greedy strategies (see, e.g., Nutini et al, 2016;Bai & Wu, 2018;Niu & Zheng, 2020;Gower et al, 2021;Zhang & Li, 2022), and others (see, e.g., Lin et al, 2015;Liu & Wright, 2016;Jiao et al, 2017). In addition, the RK method was also extended to many other problems such as the inconsistent problems (see, e.g., Zouzias & Freris, 2013;Wang et al, 2015), the ridge regression problems (see, e.g., Hefny et al, 2017;Liu & Gu, 2019), the feasibility problems (see, e.g., De Loera et al, 2017;Morshed et al, 2020Morshed et al, , 2021, etc.…”