An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions

Boţ, Radu Ioan; Csetnek, Ernö Robert; László, Szilárd Csaba

doi:10.1007/s13675-015-0045-8

Cited by 129 publications

(53 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can provide better approximation of the local Hessian at each step x k , which typically leads to improved convergence rates. There are several such metric rules proposed for both convex (Chouzenoux et al, 2014;Salzo, 2016;Lee et al, 2014) and nonconvex (Bonettini et al, 2016;Boţ et al, 2016) problems. However, despite the theoretical convergence guarantees, most such proposed rules fall short in practical cases.…”

Section: Related Workmentioning

confidence: 99%

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Park

Dhar²,

Boyd

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Variable metric proximal gradient (VM-PG) is a widely used class of convex optimization method. Lately, there has been a lot of research on the theoretical guarantees of VM-PG with different metric selections. However, most such metric selections are dependent on (an expensive) Hessian, or limited to scalar stepsizes like the Barzilai-Borwein (BB) stepsize with lots of safeguarding. Instead, in this paper we propose an adaptive metric selection strategy called the diagonal Barzilai-Borwein (BB) stepsize. The proposed diagonal selection better captures the local geometry of the problem while keeping perstep computation cost similar to the scalar BB stepsize i.e. O(n). Under this metric selection for VM-PG, the theoretical convergence is analyzed. Our empirical studies illustrate the improved convergence results under the proposed diagonal BB stepsize, specifically for ill-conditioned machine learning problems for both synthetic and real-world datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Park

Dhar²,

Boyd

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…which reads for any n ě 0 y n`1 P prox µ´1G`yn´µ´1 ∇Hpy n q˘, and is nothing else than the proximal-gradient method. An inertial version of the proximal-gradient method for solving (2.3) in the fully nonconvex setting has been considered in [12].…”

Section: The Algorithmmentioning

confidence: 99%

A Proximal Minimization Algorithm for Structured Nonconvex and Nonsmooth Problems

Boţ¹,

Csetnek²,

Nguyen³

2019

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

We propose a proximal algorithm for minimizing objective functions consisting of three summands: the composition of a nonsmooth function with a linear operator, another nonsmooth function, each of the nonsmooth summands depending on an independent block variable, and a smooth function which couples the two block variables. The algorithm is a full splitting method, which means that the nonsmooth functions are processed via their proximal operators, the smooth function via gradient steps, and the linear operator via matrix times vector multiplication. We provide sufficient conditions for the boundedness of the generated sequence and prove that any cluster point of the latter is a KKT point of the minimization problem. In the setting of the Kurdyka-Lojasiewicz property we show global convergence, and derive convergence rates for the iterates in terms of the Lojasiewicz exponent.

show abstract

“…Motivated by [11] and [13], we divide the proof into three main steps, which are listed in the following three subsections, respectively.…”

Section: The Convergence Of the Algorithmmentioning

confidence: 99%

Inertial proximal alternating minimization for nonconvex and nonsmooth problems

Zhang

2017

J Inequal Appl

View full text Add to dashboard Cite

In this paper, we study the minimization problem of the type , where f and g are both nonconvex nonsmooth functions, and R is a smooth function we can choose. We present a proximal alternating minimization algorithm with inertial effect. We obtain the convergence by constructing a key function H that guarantees a sufficient decrease property of the iterates. In fact, we prove that if H satisfies the Kurdyka-Lojasiewicz inequality, then every bounded sequence generated by the algorithm converges strongly to a critical point of L.

show abstract

An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions

Cited by 129 publications

References 36 publications

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize

A Proximal Minimization Algorithm for Structured Nonconvex and Nonsmooth Problems

Inertial proximal alternating minimization for nonconvex and nonsmooth problems

Contact Info

Product

Resources

About