Nonmonotone diagonally scaled limited-memory BFGS methods with application to compressive sensing based on a penalty model

Babaie-Kafaki, Saman; Aminifard, Zohre; Ghafoori, Saeide

doi:10.1016/j.apnum.2022.07.008

Cited by 4 publications

(5 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For standard L-BFGS the use of a diagonal seed matrix other than a scaled identity has been studied in [37,24,51,49,42,32,12,16,35,8], but convergence is usually shown for strongly convex objectives or not at all, except in [35], where global convergence is proved for the non-convex case in the sense that lim inf k→∞ ∥∇J (x k )∥ = 0. In contrast, we have lim k→∞ ∥∇J (x k )∥ = 0 in that case and a linear rate of convergence, cf.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A structured L-BFGS method with diagonal scaling and its application to image registration

Mannel,

Aggrawal

2024

Preprint

View full text Add to dashboard Cite

We devise an L-BFGS method for optimization problems in which the objective is the sum of two functions, where the Hessian of the first function is computationally unavailable while the Hessian of the second function has a computationally available approximation that allows for cheap matrix-vector products. This is a prototypical setting for many inverse problems. The proposed L-BFGS method exploits the structure of the objective to construct a more accurate Hessian approximation than in standard L-BFGS. In contrast to existing works on structured L-BFGS, we choose the first part of the seed matrix, which approximates the Hessian of the first function, as a diagonal matrix rather than a multiple of the identity. We derive two suitable formulas for the coefficients of the diagonal matrix and show that this boosts performance on real-life image registration problems, which are highly non-convex inverse problems. The new method converges globally and linearly on non-convex problems under mild assumptions in a general Hilbert space setting, making it applicable to a broad class of inverse problems. An implementation of the method is freely available. Mathematics Subject Classification 65J22 · 65K05 · 65K10 · 90C06 · 90C26 · 90C30 · 90C48 · 90C53 · 90C90

show abstract

Section: Related Workmentioning

confidence: 99%

“…4) The step size α k is, for all k, computed by Armijo with backtracking (7) or according to the WolfeŰPowell conditions (8). In the Ąrst case, we suppose in addition that there is δ > 0 such that J or ∇J is uniformly continuous in Ω δ .…”

Section: Global Convergence Of Algorithm Rosementioning

confidence: 99%

A structured L-BFGS method with diagonal scaling and its application to image registration

Mannel,

Aggrawal

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…where t > 0 is the Dai-Liao parameter [19]. In another attempt, Babaie-Kafaki et al [11] suggested the following penalized version of (2.5): min…”

Section: A Diagonal Admm-based Quasi-newton Updatementioning

confidence: 99%

“…It can be observed that MMLBFGS and MLMBFGS generate descent directions regardless of the line search. Furthermore, we consider the diagonal QN methods proposed in [6], [11] and [26] in our comparisons, here respectively named by DQNBN1, DQNBN2 and DQNMSG. For DQNADMM, we set ρ = 10 and µ 0 = 1 in (2.9).…”

Section: Numerical Experimentsmentioning

confidence: 99%

“…Recently, the QN methods have been much heeded in practical applications such as image processing, time series prediction, neural networks training, document categorization, managing demands in the water distribution networks, machine learning, robotics, solving systems of nonlinear equations, curve fitting by B-splines, matrix approximation in Frobenius norm, computation of the matrix geometric mean and estimating unitary symmetric eigenvalues of the complex tensors; for more details see [11] and references therein. The methods have also been well-combined with classical optimization tools such as conjugate gradient methods [12] as well as metaheuristic algorithms [32].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Nonmonotone Admm-Based Diagonal Quasi-Newton Update With Application to the Compressive Sensing Problem

Aminifard,

Babaie-Kafaki

2023

Mathematical Modelling and Analysis

Self Cite

View full text Add to dashboard Cite

Considering a minimization problem according to the Byrd-Nocedal measure function together with the secant equation, a diagonal quasi-Newton updating formula is suggested. To find the optimal elements of the updating matrix, the well-known algorithm of the alternating direction method of multipliers (ADMM) is employed. Moreover, convergence analysis is conducted based on a modified nonmonotone Armijo line search incorporating the simulated annealing strategy. Lastly, performance of the method is numerically tested on a set of CUTEr functions and on a smooth transcendental approximation of the compressive sensing problem. Across the computational spectrum, the given method turns out to be successful.

show abstract

Phase stability and elastic properties of Al–Pr intermetallic compounds from first principles calculations

Ling

Liu

Zou

et al. 2023

Physica B: Condensed Matter

View full text Add to dashboard Cite

Nonmonotone diagonally scaled limited-memory BFGS methods with application to compressive sensing based on a penalty model

Cited by 4 publications

References 61 publications

A structured L-BFGS method with diagonal scaling and its application to image registration

A structured L-BFGS method with diagonal scaling and its application to image registration

A Nonmonotone Admm-Based Diagonal Quasi-Newton Update With Application to the Compressive Sensing Problem

Phase stability and elastic properties of Al–Pr intermetallic compounds from first principles calculations

Contact Info

Product

Resources

About