Diagonally scaled memoryless quasi–Newton methods with application to compressed sensing

Abstract

<p style='text-indent:20px;'>Memoryless quasi–Newton updating formulas of BFGS (Broyden–Fletcher–Goldfarb–Shanno) and DFP (Davidon–Fletcher–Powell) are scaled using well-structured diagonal matrices. In the scaling approach, diagonal elements as well as eigenvalues of the scaled memoryless quasi–Newton updating formulas play significant roles. Convergence analysis of the given diagonally scaled quasi–Newton methods is discussed. At last, performance of the methods is numerically tested on a set of CUTEr problems as well as the compressed sensing problem.</p>

A scalar Hessian estimation with a sparse nonmonotone line search technique for the sparse recovery problem

“…Also, NMSGP outperforms SNMSBB, denoting the superiority of the parameter (12). 4. Concluding remarks and possible future works.…”

“…Now, we refer to the general smooth unconstrained optimization problem to acquire another suitable shrinkage step length. As known, the scaled memoryless quasi-Newton (QN) algorithms reveal potential worthiness amidst the various methodologies to address the unconstrained optimization models [4,[25][26][27]. At the core of the QN algorithms, the BFGS method with the scaled memoryless updating formula…”

A Nonmonotone Admm-Based Diagonal Quasi-Newton Update With Application to the Compressive Sensing Problem

“…In another attempt, Andrei [6] proposed a diagonal QN update by minimizing the measure function of Byrd and Nocedal [17], based on forward and central finite differences [7]. A family of diagonal QN updates was suggested in [3] in accordance to the DFP (Davidon-Fletcher-Powell) and the BFGS (Broyden-Fletcher-Goldfarb-Shanno) updating formulas. A tridiagonal Hessian approximation has also been developed in [9].…”