A Sampling Kaczmarz--Motzkin Algorithm for Linear Feasibility

Loera, Jesús A. De; Haddock, Jamie; Needell, Deanna

doi:10.1137/16m1073807

Cited by 74 publications

(110 citation statements)

References 58 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…In each iteration of SDA, a dual variable is updated by choosing a point in a subspace spanned by the columns of a random matrix drawn independently from a fixed distribution. In [13], the authors combined the relaxation method of Motzkin [15] (also known as Kaczmarz method with the "most violated constraint control") and the randomized Kaczmarz method [23] to obtain a family of algorithms called Sampling Kaczmarz-Motzkin (SKM) for solving the linear systems Ax ≤ b. In SKM, at each time a subset of inequalities are picked, and the variables are updated based on the projection to the subspace corresponding to the most violated linear equality/inequality.…”

Section: Background On the Convergence Of G-s Type Algorithmmentioning

confidence: 99%

A linearly convergent doubly stochastic Gauss–Seidel algorithm for solving linear equations and a certain class of over-parameterized optimization problems

et al. 2019

View full text Add to dashboard Cite

Consider the classical problem of solving a general linear system of equations Ax = b. It is well known that the (successively over relaxed) Gauss-Seidel scheme and many of its variants may not converge when A is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G-S type algorithm that works for any A? In this paper we answer this question affirmatively by proposing a doubly stochastic G-S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a nonuniform double stochastic scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem Ax ≤ b with an arbitrary A, as well as high-dimensional minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G-S algorithm converge. * equal contributions. M. Razaviyayn is with the

show abstract

Section: Background On the Convergence Of G-s Type Algorithmmentioning

confidence: 99%

A linearly convergent doubly stochastic Gauss–Seidel algorithm for solving linear equations and a certain class of over-parameterized optimization problems

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In the context of Kaczmarz type methods, recently Loizou et. al [30] analyzed the so-called momentum induced GR sketching method [17] for 5 The difference between the Kaczmarz method for linear system and linear feasibility is that for the case of linear systems we use 6 Recent works show that instead of orthogonal projection one can choose the projection parameter δ between 0 and 2 [10,34] (i.e., given x k , set…”

Section: Sampling Kaczmarz-motzkin (Skm)mentioning

confidence: 99%

“…with E S denotes the required expectation corresponding to the sampling distribution S. The above expectation expression was first used by De Loera et.al in their work [10] to analyze the SKM method. To simplify the above expectation expression, let us define the function f : R n → R and the gradient of f as follows:…”

Section: Technical Toolsmentioning

confidence: 99%

“…Proof Take, β = m in Theorem 4.6. Then using the definition, [10]) Let {x k } be the sequence of random iterates generated by algorithm 1 with γ = 0 (SKM method) starting with x 0 ∈ R n . With 0 < δ < 2, the sequence of iterates {x k } converges and the following result holds:…”

Section: Also the Average Iteratexmentioning

confidence: 99%

“…Recent advances in the area of iterative algorithms suggest that randomization can produce theoretically rigorous and computationally efficient projection algorithms for solving many computational problems such as linear feasibility, linear systems and convex optimization problems, etc. [58,26,37,11,61,25,32,17,51,38,10,54,34]. In the following, we discuss some of the classical and modern algorithmic developments over the years for solving large-scale linear feasibility problems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Accelerated sampling Kaczmarz Motzkin algorithm for the linear feasibility problem

2019

View full text Add to dashboard Cite

The Sampling Kaczmarz Motzkin (SKM) algorithm is a generalized method for solving large-scale linear systems of inequalities. Having its root in the relaxation method of Agmon, Schoenberg, and Motzkin and the randomized Kaczmarz method, SKM outperforms the state-of-the-art methods in solving large-scale Linear Feasibility (LF) problems. Motivated by SKM's success, in this work, we propose an Accelerated Sampling Kaczmarz Motzkin (ASKM) algorithm which achieves better convergence compared to the standard SKM algorithm on ill-conditioned problems. We provide a thorough convergence analysis for the proposed accelerated algorithm and validate the results with various numerical experiments. We compare the performance and effectiveness of ASKM algorithm with SKM, Interior Point Method (IPM) and Active Set Method (ASM) on randomly generated instances as well as Netlib LPs. In most of the test instances, the proposed ASKM algorithm outperforms the other state-of-the-art methods.

show abstract

Greedy Motzkin–Kaczmarz methods for solving linear systems

Zhang¹,

2021

Numerical Linear Algebra App

View full text Add to dashboard Cite

The famous greedy randomized Kaczmarz (GRK) method uses the greedy selection rule on maximum distance to determine a subset of the indices of working rows. In this paper, with the greedy selection rule on maximum residual, we propose the greedy randomized Motzkin-Kaczmarz (GRMK) method for linear systems. The block version of the new method is also presented. We analyze the convergence of the two methods and provide the corresponding convergence factors. The computational complexities of the proposed methods are also given.Extensive numerical experiments show that the GRMK method has almost the same performance as the GRK method for dense matrices and the former performs better in computing time for some sparse matrices and some realistic practical problems, and their block versions have almost the same performance for most of cases.

show abstract

A Sampling Kaczmarz--Motzkin Algorithm for Linear Feasibility

Cited by 74 publications

References 58 publications

A linearly convergent doubly stochastic Gauss–Seidel algorithm for solving linear equations and a certain class of over-parameterized optimization problems

A linearly convergent doubly stochastic Gauss–Seidel algorithm for solving linear equations and a certain class of over-parameterized optimization problems

Accelerated sampling Kaczmarz Motzkin algorithm for the linear feasibility problem

Greedy Motzkin–Kaczmarz methods for solving linear systems

Contact Info

Product

Resources

About