Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements

Chen, Xuemei; Powell, Alexander M.

doi:10.1007/s00041-012-9237-2

Cited by 46 publications

(37 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…as the scaled condition number of X. This result was extended to the inconsistent case [16], derived via a probabilistic almost-sure convergence perspective [2], accelerated in multiple ways [5,4,22,19,18], and generalized to other settings [13,23,17].…”

Section: Randomized Kaczmarz (Rk) For Xβ = Ymentioning

confidence: 96%

Rows versus Columns: Randomized Kaczmarz or Gauss--Seidel for Ridge Regression

Hefny¹,

Needell²,

Ramdas³

2017

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

The Kaczmarz and Gauss-Seidel methods aim to solve a m × n linear system Xβ = y by iteratively refining the solution estimate; the former uses random rows of X to update β given the corresponding equations and the latter uses random columns of X to update corresponding coordinates in β. Recent work analyzed these algorithms in a parallel comparison for the overcomplete and undercomplete systems, showing convergence to the ordinary least squares (OLS) solution and the minimum Euclidean norm solution respectively. This paper considers the natural follow-up to the OLS problem -ridge or Tikhonov regularized regression. By viewing them as variants of randomized coordinate descent, we present variants of the randomized Kaczmarz (RK) and randomized Gauss-Siedel (RGS) for solving this system and derive their convergence rates. We prove that a recent proposal, which can be interpreted as randomizing over both rows and columns, is strictly suboptimal -instead, one should always work with randomly selected columns (RGS) when m > n (#rows > #cols) and with randomly selected rows (RK) when n > m (#cols > #rows). * Authors are listed in alphabetical order. arXiv:1507.05844v2 [math.NA]

show abstract

Section: Randomized Kaczmarz (Rk) For Xβ = Ymentioning

confidence: 96%

Rows versus Columns: Randomized Kaczmarz or Gauss--Seidel for Ridge Regression

Hefny¹,

Needell²,

Ramdas³

2017

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

show abstract

“…Kaczmarz bounds can similarly be used to control rates of almost sure convergence. The next theorem can be proven in the same manner as either one of the two different proofs of Theorem 6.2 in [11].…”

Section: )mentioning

confidence: 81%

“…(1) As necessary background lemmas, the error bounds of [11] are extended to the setting of fusion frames and, in particular, they provide bounds on error moments and almost sure convergence rates for the algorithm (1.5). (2) We address the following main question.…”

Section: 2mentioning

confidence: 99%

Randomized Subspace Actions and Fusion Frames

Powell

2015

Constr Approx

Self Cite

View full text Add to dashboard Cite

A randomized subspace action algorithm is investigated for fusion frame signal recovery problems and for the problem of recovering a signal from projections onto random subspaces. It is noted that Kaczmarz bounds provide upper bounds on the algorithm's error moments. The main question of which probability distributions on a random subspace lead to provably fast convergence is addressed. In particular, it is proven which distributions give minimal Kaczmarz bounds, and hence give best control on error moment upper bounds arising from Kaczmarz bounds. Uniqueness of the optimal distributions is also addressed.

show abstract

“…This work was extended to the inconsistent case in [Nee10], which shows linear convergence to within some fixed radius of the least squares solution. The almost-sure guarantees were recently derived by Chen and Powell [CP12]. To break the convergence barrier, relaxation parameters can be introduced, so that each iterate is over or under projected onto each solution space.…”

Section: Related Work and Discussionmentioning

confidence: 99%

Randomized block Kaczmarz method with projection for solving least squares

Needell

Zhao

Zouzias

2015

Linear Algebra and its Applications

141

107

View full text Add to dashboard Cite

The Kaczmarz method is an iterative method for solving overcomplete linear systems of equations Ax = b. The randomized version of the Kaczmarz method put forth by Strohmer and Vershynin iteratively projects onto a randomly chosen solution space given by a single row of the matrix A and converges linearly 1 in expectation to the solution of a consistent system. In this paper we analyze two block versions of the method each with a randomized projection, designed to converge in expectation to the least squares solution, often faster than the standard variants. Our approach utilizes both a row and column-paving of the matrix A to guarantee linear convergence when the matrix has consistent row norms (called nearly standardized), and a single column-paving when the row norms are unbounded. The proposed methods are an extension of the block Kaczmarz method analyzed by Needell and Tropp and the Randomized Extended Kaczmarz method of Zouzias and Freris. The contribution is thus two-fold; unlike the standard Kaczmarz method, our results demonstrate convergence to the least squares solution of inconsistent systems (both methods in the nearly standardized case and the second method in other cases). By using appropriate blocks of the matrix this convergence can be significantly accelerated, as is demonstrated by numerical experiments.

show abstract

Almost Sure Convergence of the Kaczmarz Algorithm with Random Measurements

Cited by 46 publications

References 19 publications

Rows versus Columns: Randomized Kaczmarz or Gauss--Seidel for Ridge Regression

Rows versus Columns: Randomized Kaczmarz or Gauss--Seidel for Ridge Regression

Randomized Subspace Actions and Fusion Frames

Randomized block Kaczmarz method with projection for solving least squares

Contact Info

Product

Resources

About