Tight upper bounds for the convergence of the randomized extended Kaczmarz and Gauss–Seidel algorithms

Du, Kun

doi:10.1002/nla.2233

Cited by 78 publications

(59 citation statements)

References 31 publications

(85 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We denote the columns and rows of A by

{A_{j}}_{j = 1}^{n}

and

{a_{i}^{T}}_{i = 1}^{m}

, respectively. Then, the iteration schemes (16) and (17) become

\begin{align} ({, \begin{matrix} leftsubarray \\ array z^{(k)} = (I_{m} - \frac{A_{j} A_{j}^{T}}{‖ A_{j} ‖^{2}}) z^{(k - 1)}, \\ array x^{(k)} = x^{(k - 1)} + \frac{{[b - z^{(k)}]}_{i} - a_{i}^{T} x^{(k - 1)}}{‖ a_{i} ‖^{2}} a_{i}, \end{matrix}) \end{align}

respectively, which is the REK‐D iteration scheme, see Algorithm 3 of Reference 20. Note that when the initial guesses are x (0) = 0 and z (0) = b , and we update iteration vector x ( k ) using the vector z ( k − 1) , then we get REK‐ZF method, see Algorithm 3 of Reference 19.…”

Section: The Extended Rri Methods and Its Convergencementioning

confidence: 99%

“…To the best of our knowledge, the BREK-D, GEK, and BGEK methods are new. Note that if we introduce the symmetric positive definite matricesĜ andG instead of the identity matrices in (18), (20), (19), and (21), we can obtain more general iterative schemes of REK-D, GEK, BREK-D, and BGEK methods, respectively. Correspondingly, in these cases we use the matrices…”

Section: The Erri Iterationmentioning

confidence: 99%

“…Recently, its modified variant is given by Algorithm 3 of Du. 20 To avoid confusion, we refer to these two methods as REK-ZF and REK-D, respectively, for the rest of this paper. It is shown that the REK-D method enjoys a tighter upper bound for its convergence rate in expectation than the REK-ZF method, see remark 1 of Reference 20.…”

Section: Introductionmentioning

confidence: 99%

“…When the linear system (1) is overdetermined and inconsistent, based on the work of Popa in References 16‐18, Zouzias and Freris propose a randomized iterative least squares (LS) solver for this problem, which is a randomized variant of extended Kaczmarz method, see Algorithm 3 of Reference 19, and prove that the iteration vector exponentially converges in expectation to the least squares solution. Recently, its modified variant is given by Algorithm 3 of Du 20 . To avoid confusion, we refer to these two methods as REK‐ZF and REK‐D, respectively, for the rest of this paper.…”

Section: Introductionmentioning

confidence: 99%

“…To avoid confusion, we refer to these two methods as REK‐ZF and REK‐D, respectively, for the rest of this paper. It is shown that the REK‐D method enjoys a tighter upper bound for its convergence rate in expectation than the REK‐ZF method, see remark 1 of Reference 20. Besides, Needell, Zhao and Zouzias 13 present a block version of the REK‐ZF method, which is often faster than the standard variant.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Semiconvergence analysis of the randomized row iterative method and its extended variants

Xiang

2020

Numerical Linear Algebra App

View full text Add to dashboard Cite

The row iterative method is popular in solving the large-scale ill-posed problems due to its simplicity and efficiency. In this work we consider the randomized row iterative (RRI) method to tackle this issue. First, we present the semiconvergence analysis of RRI method for the overdetermined and inconsistent system, and derive upper bounds for the noise error propagation in the iteration vectors. To achieve a least squares solution, we then propose an extended version of the RRI (ERRI) method, which in fact can converge in expectation to the solution of the overdetermined or underdetermined, consistent or inconsistent systems. Finally, some numerical examples are given to demonstrate the convergence behaviors of the RRI and ERRI methods for these types of linear system.

show abstract

“…We denote the columns and rows of A by

{A_{j}}_{j = 1}^{n}

and

{a_{i}^{T}}_{i = 1}^{m}

, respectively. Then, the iteration schemes (16) and (17) become

\begin{align} ({, \begin{matrix} leftsubarray \\ array z^{(k)} = (I_{m} - \frac{A_{j} A_{j}^{T}}{‖ A_{j} ‖^{2}}) z^{(k - 1)}, \\ array x^{(k)} = x^{(k - 1)} + \frac{{[b - z^{(k)}]}_{i} - a_{i}^{T} x^{(k - 1)}}{‖ a_{i} ‖^{2}} a_{i}, \end{matrix}) \end{align}

Section: The Extended Rri Methods and Its Convergencementioning

confidence: 99%

Section: The Erri Iterationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Semiconvergence analysis of the randomized row iterative method and its extended variants

Xiang

2020

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

On constrained Kaczmarz algorithm with momentum for image reconstruction

Yin,

Ouyang

2024

Math Methods in App Sciences

View full text Add to dashboard Cite

SummaryThe Kaczmarz algorithm is widely used in image reconstruction. In this paper, we propose an enhanced version of the Kaczmarz algorithm by integrating Nesterov momentum into the Kaczmarz algorithm, denoted as the Kaczmarz algorithm with momentum. Our innovation offers a novel approach to accelerate the convergence rate of the Kaczmarz algorithm significantly. Based on greedy rules for selecting work row, we prove that the Kaczmarz method with momentum can achieve global linear convergence rates. Furthermore, we extend a constrained version of the Kaczmarz algorithm with Nesterov momentum for improved image reconstruction and prove its convergence. Numerical experiments show that the proposed algorithm outperforms the original Kaczmarz algorithm in terms of efficiency for image reconstruction.

show abstract

Greedy Motzkin–Kaczmarz methods for solving linear systems

Zhang¹,

2021

Numerical Linear Algebra App

View full text Add to dashboard Cite

The famous greedy randomized Kaczmarz (GRK) method uses the greedy selection rule on maximum distance to determine a subset of the indices of working rows. In this paper, with the greedy selection rule on maximum residual, we propose the greedy randomized Motzkin-Kaczmarz (GRMK) method for linear systems. The block version of the new method is also presented. We analyze the convergence of the two methods and provide the corresponding convergence factors. The computational complexities of the proposed methods are also given.Extensive numerical experiments show that the GRMK method has almost the same performance as the GRK method for dense matrices and the former performs better in computing time for some sparse matrices and some realistic practical problems, and their block versions have almost the same performance for most of cases.

show abstract

Tight upper bounds for the convergence of the randomized extended Kaczmarz and Gauss–Seidel algorithms

Cited by 78 publications

References 31 publications

Semiconvergence analysis of the randomized row iterative method and its extended variants

Semiconvergence analysis of the randomized row iterative method and its extended variants

On constrained Kaczmarz algorithm with momentum for image reconstruction

Greedy Motzkin–Kaczmarz methods for solving linear systems

Contact Info

Product

Resources

About