Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

Schöpfer, Frank; Lorenz, Dirk A.; Tondji, Lionel; Winkler, Maximilian

doi:10.48550/arxiv.2201.08620

Cited by 2 publications

(5 citation statements)

References 25 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fixing the relaxation parameters w k,i = 1 for all iterations k and indices i lead to the standard RSK method. In [23], an extension of the RSK with linear expected convergence has been proposed for solving sparse least squares and impulsive noise problems while requiring only one additional column of the system matrix in each iteration. For consistent systems the iterates of the standard RSK method converge in expectation to the solution of the regularized Basis Pursuit Problem…”

Section: Related Workmentioning

confidence: 99%

Faster Randomized Block Sparse Kaczmarz by Averaging

Tondji¹,

Lorenz²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The standard randomized sparse Kaczmarz (RSK) method is and algorithm to compute sparse solutions of linear systems of equations and uses sequential updates, and thus, does not take advantage of parallel computations. In this work, we introduce a parallel (mini batch) version of RSK based on averaging several Kaczmarz steps. Naturally, this method allows for parallelization and we show that it can also leverage large over-relaxation. We prove linear expected convergence and show that, given that parallel computations can be exploited, the method provably provides faster convergence than the standard method. This method can also be viewed as a variant of the linearized Bregman algorithm, a randomized dual block coordinate descent update, a stochastic mirror descent update, or a relaxed version of RSK and we recover the standard RSK method when the batch size is equal to one. We also provide estimates for inconsistent systems and show that the iterates convergence to an error in the order of the noise level. Finally, numerical examples illustrate the benefits of the new algorithm.

show abstract

Section: Related Workmentioning

confidence: 99%

Faster Randomized Block Sparse Kaczmarz by Averaging

Tondji¹,

Lorenz²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Like the algorithms in the works [24,13,5,23,20], our approach combines two randomized iterative algorithms. Specifically, for the consistent case (b ∈ range(AB)), we propose using the RK algorithm to solve the subsystem Ay = b followed by the RRK algorithm to solve the minimization problem (3) as shown in Algorithm 4, and call it the RK-RRK algorithm.…”

Section: The Proposed Algorithmsmentioning

confidence: 99%

“…x 2 2 + λ x 1 , we call the resulting algorithm the RK-RSK algorithm. In the following remark, we give the relationship between the ExSRK algorithm [20] and the RK-RSK algorithm.…”

Section: The Rk-rrk Algorithm For the Case B ∈ Range(ab)mentioning

confidence: 99%

“…In many applications, the matrix B in (1) acts as a frame or redundant dictionary, and the system (1) may have a sparse solution (usually not the minimum norm one). If the matrix C = AB is explicitly given, the well known randomized sparse Kaczmarz (RSK) algorithm [12,16,19] can be used to find a sparse solution for the consistent case, and the recently proposed extended sparse randomized Kaczmarz (ExSRK) algorithm [20] can be used to find a sparse least squares solution for the inconsistent case. It was shown in [19] that for the consistent case the RSK algorithm converges linearly to the unique solution of the regularized basis pursuit problem minimize 1 2…”

Section: Introductionmentioning

confidence: 99%

“…Cx = b. It was shown in [20] that the ExSRK algorithm converges linearly to the unique solution of the combined optimization problem…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Regularized randomized iterative algorithms for factorized linear systems

Du¹

2022

Preprint

View full text Add to dashboard Cite

Randomized iterative algorithms for solving a factorized linear system, ABx = b with A ∈ R m× , B ∈ R ×n , and b ∈ R m , have recently been proposed. They take advantage of the factorized form and avoid forming the matrix C = AB explicitly. However, they can only find the minimum norm (least squares) solution. In contrast, the regularized randomized Kaczmarz (RRK) algorithm can find solutions with certain structures from consistent linear systems. In this work, by combining the randomized Kaczmarz algorithm or the randomized Gauss-Seidel algorithm with the RRK algorithm, we propose two novel regularized randomized iterative algorithms to find (least squares) solutions with certain structures of ABx = b. We prove linear convergence of the new algorithms. Computed examples are given to illustrate that the new algorithms can find sparse (least squares) solutions of ABx = b and can be better than the existing randomized iterative algorithms for the corresponding full linear system Cx = b with C = AB.

show abstract

Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

Cited by 2 publications

References 25 publications

Faster Randomized Block Sparse Kaczmarz by Averaging

Faster Randomized Block Sparse Kaczmarz by Averaging

Regularized randomized iterative algorithms for factorized linear systems

Contact Info

Product

Resources

About