Partial-Quasi-Newton Methods

Liu, Chengchang; Bi, Shuxian; Luo, Luo; Lui, John C. S.

doi:10.1145/3534678.3539379

Cited by 2 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use the Euclidean norm λ(z) F(z) 2 to measure the convergence of our algorithm. The advantage of block updates in SR-k updates results a faster superlinear convergence than Liu and Luo [22]'s methods. Following the analysis of SR-k methods for convex optimization, we obtain the results for solving nonlinear equations as follows.…”

Section: A the Proofs In Sectionmentioning

confidence: 98%

“…where f : R d → R is smooth and strongly self-concordant. Quasi-Newton methods [2,3,4,6,8,33,36] are widely recognized for their fast convergence rates and efficient updates, which attracts growing attention in many fields such as statistics [1,16,37], economics [20,24] and machine learning [12,15,19,22,23]. Unlike standard Newton methods which need to compute the Hessian and its inverse, quasi-Newton methods go along the descent direction by the following scheme…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Symmetric Rank-$k$ Methods

Liu¹,

Chen²,

Luo³

2023

Preprint

View full text Add to dashboard Cite

This paper proposes a novel class of block quasi-Newton methods for convex optimization which we call symmetric rank-k (SR-k) methods. Each iteration of SR-k incorporates the curvature information with k Hessian-vector products achieved from the greedy or random strategy. We prove SR-k methods have the local superlinear convergence rate of O (1 − k/d) t(t−1)/2 for minimizing smooth and strongly self-concordant function, where d is the problem dimension and t is the iteration counter. This is the first explicit superlinear convergence rate for block quasi-Newton methods and it successfully explains why block quasi-Newton methods converge faster than standard quasi-Newton methods in practice.

show abstract

Section: A the Proofs In Sectionmentioning

confidence: 98%