Randomized Extended Average Block Kaczmarz for Solving Least Squares

Randomized regularized extended Kaczmarz algorithms for tensor recovery

“…Next we analyze the convergence of the RREK algorithm. Our analysis is similar to that of [9], but slightly more complicated. The convergence estimates depend on the positive numbers λ r and λ c defined as…”

“…Numerical experiments on a tensor least squares problem and a sparse tensor recovery problem confirm the theoretical results. In the future, we will use the existing acceleration strategies such as those in [25,9,4,31] to further improve the efficiency when applied to large scale problems. Moreover, we note that the tensor recovery model ( 2) can be used as a variable selection procedure and we are studying its performance compared with the elastic net [34] and the lasso [30].…”

“…In the last inequality, we use the facts that 2α c − α 2 c λ c > 0, E (k) ∈ range K (A) (by induction), and (9). Next, by the law of total expectation, we have…”

“…The solution X of ( 2) is a least squares solution of the inconsistent system A * X = B with some desirable characteristics promoted by regularization terms of the objective function f . In recent years, randomized iterative algorithms for linear systems of equations with massive data sets have been greatly developed due to low memory footprints and good numerical performance, such as the randomized Kaczmarz (RK) algorithm [29], the randomized coordinate descent (RCD) algorithm [16], the randomized extended Kaczmarz (REK) algorithm [35], and their extensions, e.g., [24,10,20,25,1,2,3,8,22,9,4,31,32,11,33]. The RK algorithm converges linearly in expectation to a solution of consistent linear systems [29,35] and to within a radius (convergence horizon) of a (least squares) solution of inconsistent linear systems [23].…”

Greedy Randomized and Maximal Weighted Residual Kaczmarz Methods with Oblique Projection

“…Strohmer and Vershynin [42] proposed a randomized Kaczmarz (RK) method which selects a given row with proportional to the Euclidean norm of the rows of the coefficient matrix A, and proved its convergence. After the above work, research on the Kaczmarz-type methods was reignited recently, see for example, the randomized block Kaczmarz-type methods [33,35,31,12], the greedy version of Kaczmarz-type methods [25,49,47,1,2], the extended version of Kaczmarz-type methods [29,4,10], and many others [24,14,11,38,34,41]. Kaczmarz's research also accelerated the development of column action iterative methods represented by the coordinate descent method [21].…”