Extended randomized Kaczmarz method for sparse least squares and impulsive noise problems

Schöpfer, Frank; Lorenz, Dirk A.; Tondji, Lionel; Winkler, Maximilian

doi:10.1016/j.laa.2022.07.003

Cited by 15 publications

(10 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is unfavorable when the size of A is large. For more variants and studies of row-action iterative methods, we refer to [4,5,13,17,28,30,33] and the references therein.…”

Section: The Randomized Multiple Row-action Methodsmentioning

confidence: 99%

On the randomized multiple row-action methods for solving linear least-squares problems

Wu,

Liu,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

The randomized row-action method is a popular representative of the iterative algorithm because of its efficiency in solving the overdetermined and consistent systems of linear equations. In this paper, we present an extended randomized multiple row-action method to solve a given overdetermined and inconsistent linear system and analyze its computational complexities at each iteration. We prove that the proposed method can linearly converge in the mean square to the least-squares solution with a minimum Euclidean norm. Several numerical studies are presented to corroborate our theoretical findings. The real-world applications, such as image reconstruction and large noisy data fitting in computer-aided geometric design, are also presented for illustration purposes. AMS subject classifications: 65F10, 65F20, 65F25, 65D10, 68W20

show abstract

“…It is unfavorable when the size of A is large. For more variants and studies of row-action iterative methods, we refer to [4,5,13,17,28,30,33] and the references therein.…”

Section: The Randomized Multiple Row-action Methodsmentioning

confidence: 99%

On the randomized multiple row-action methods for solving linear least-squares problems

Wu,

Liu,

Wang

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…If we want to compute minimum 2-norm solutions of the least squares problem min x∈R n ||Ax − b|| 2 2 , the extended Kaczmarz method can be used [3]. If the noise is concentrated in the complement of the range of the system matrix A, that is, e ∈ (A) ⟂ ∩ (𝜏), the extended sparse Kaczmarz method has been proposed and analyzed in Schöpfer [37]. However, when e ∉ (A) ⟂ , we do not know how to modify the extended sparse Kaczmarz algorithm to compute minimum 1-norm solutions to noisy linear systems.…”

Section: Convergencementioning

confidence: 99%

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

Niu,

Zheng

2024

Math Methods in App Sciences

View full text Add to dashboard Cite

The randomized sparse Kaczmarz (RSK) method is an algorithm used to calculate sparse solutions for the basis pursuit problem. In this paper, we propose an algorithm framework for computing sparse solutions of linear systems, which includes the sparse Kaczmarz and sparse block Kaczmarz algorithms. In order to overcome the limitations of the penalty, we design an effective and new randomized sparse Kaczmarz algorithm (RSK‐MCP) based on the non‐convex minimax‐concave penalty (MCP) in sparse signal reconstruction. Additionally, we prove that the RSK‐MCP algorithm is equivalent to the randomized coordinate descent method for the corresponding dual problem. Based on this result, we demonstrate that the RSK‐MCP algorithm exhibits linear convergence, meaning it converges to a sparse solution of the MCP model when the regularization of MCP is a strongly convex function. Numerical experiments indicate that the RSK‐MCP algorithm outperforms RSK‐L1 in terms of both efficiency and accuracy.

show abstract

“…Here, the notion of projection was replaced by the (more general) Bregman projection, giving rise to the 'sparse' Kaczmarz method, which can find sparse solutions of the system. The method has been further extended to inconsistent systems [53] and accelerated by block averaging [56]. But so far only linear systems have been addressed.…”

Section: Related Work and Our Contributionsmentioning

confidence: 99%

A Bregman-Kaczmarz method for nonlinear systems of equations

Gower¹,

Lorenz²,

Winkler³

2023

Preprint

View full text Add to dashboard Cite

We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman projections onto the solution space of a Newton equation. In the special case of euclidean projections, the method is known as nonlinear Kaczmarz method. Furthermore if the component functions are nonnegative, we are in the setting of optimization under the interpolation assumption and the method reduces to SGD with the recently proposed stochastic Polyak step size. For general Bregman projections, our method is a stochastic mirror descent with a novel adaptive step size. We prove that in the convex setting each iteration of our method results in a smaller Bregman distance to exact solutions as compared to the standard Polyak step. Our generalization to Bregman projections comes with the price that a convex one-dimensional optimization problem needs to be solved in each iteration. This can typically be done with globalized Newton iterations. Convergence is proved in two classical settings of nonlinearity: for convex nonnegative functions and locally for functions which fulfill the tangential cone condition. Finally, we show examples in which the proposed method outperforms similar methods with the same memory requirements.

show abstract

Extended randomized Kaczmarz method for sparse least squares and impulsive noise problems

Cited by 15 publications

References 35 publications

On the randomized multiple row-action methods for solving linear least-squares problems

On the randomized multiple row-action methods for solving linear least-squares problems

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

A Bregman-Kaczmarz method for nonlinear systems of equations

Contact Info

Product

Resources

About