Restricted strong convexity and its applications to convergence analysis of gradient-type methods in convex optimization

Zhang, Hui; Cheng, Lizhi

doi:10.1007/s11590-014-0795-x

Cited by 36 publications

(37 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those sorts of relaxations were further exploited in [6,7] (relaxation of the strong convexity requirement, with motivational examples). We leave further investigations in that direction for future research.…”

Section: Upper Bounds On the Global Convergence Ratesmentioning

confidence: 99%

“…Other related research trends feature linear convergence rates for the (proximal) gradient method under weaker assumptions than strong convexity, and linear convergence rates under inexact first-order information [5]. Among others, restricted strong convexity-type results are presented in [6,7], and convergence under the Polyak-Lojasiewicz condition were very recently presented in [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Taylor

Hendrickx

Glineur

2018

J Optim Theory Appl

View full text Add to dashboard Cite

We study the worst-case convergence rates of the proximal gradient method for minimizing the sum of a smooth strongly convex function and a non-smooth convex function, whose proximal operator is available.We establish the exact worst-case convergence rates of the proximal gradient method in this setting for any step size and for different standard performance measures: objective function accuracy, distance to optimality and residual gradient norm.The proof methodology relies on recent developments in performance estimation of first-order methods, based on semidefinite programming. In the case of the proximal gradient method, this methodology allows obtaining exact and non-asymptotic worst-case guarantees, that are conceptually very simple, although apparently new.On the way, we discuss how strong convexity can be replaced by weaker assumptions, while preserving the corresponding convergence rates. We also establish that the same fixed step size policy is optimal for all three performance measures. Finally, we extend recent results on the worst-case behavior of gradient descent with exact line search to the proximal case.

show abstract

Section: Upper Bounds On the Global Convergence Ratesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Taylor

Hendrickx

Glineur

2018

J Optim Theory Appl

View full text Add to dashboard Cite

show abstract

“…We finally utilize the following lemma to obtain an alternative form of the restricted strong convexity condition [69].…”

Section: Proof Of the Regularity Conditionmentioning

confidence: 99%

A Generalization of Wirtinger Flow for Exact Interferometric Inversion

Yonel¹,

Yazıcı²

2019

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

Interferometric inversion involves recovery of a signal from cross-correlations of its linear transformations. A close relative of interferometric inversion is the generalized phase retrieval problem, which consists of recovering a signal from the auto-correlations of its linear transformations. Recently, significant advancements have been made in phase retrieval methods despite the ill-posed, and non-convex nature of the problem. One such method is Wirtinger Flow (WF) [1], a non-convex optimization framework that provides high probability guarantees of exact recovery under certain measurement models, such as coded diffraction patterns, and Gaussian sampling vectors. In this paper, we develop a generalization of WF for interferometric inversion, which we refer to as Generalized Wirtinger Flow (GWF). GWF theory extends the probabilistic exact recovery results in [1] to arbitrary measurement models characterized in the equivalent lifted problem, hence covers a larger class of measurement models. GWF framework unifies the theory of low rank matrix recovery (LRMR) and the non-convex optimization approach of WF, thereby establishes theoretical advantages of the non-convex approach over LRMR. We show that the conditions for exact recovery via WF can be derived through a low rank matrix recovery formulation. We identify a new sufficient condition on the lifted forward model that directly implies exact recovery conditions of standard WF. This condition is less stringent than those of LRMR, which is the state of the art approach for exact interferometric inversion. We next establish our sufficient condition for the cross-correlations of linear measurements collected by complex Gaussian sampling vectors. In the particular case of the Gaussian model, we show that the exact recovery conditions of standard WF imply our sufficient condition, and that the regularity condition of WF is redundant for the interferometric inversion problem. Finally, we demonstrate the effectiveness of GWF numerically in a deterministic multi-static radar imaging scenario.

show abstract

“…In the next theorem, we show that (2.5) is also a sufficient condition for linear convergence (with σ = 0) of the stochastic gradient method for the class of restricted strongly convex function f . Restricted strong convexity is much weaker than strong convexity, some examples and properties of restricted strongly convex functions can be found in [17]. Note that if f is a strongly convex function, A is a linear mapping, then the composite function f • A is restricted strongly convex.…”

Section: )mentioning

confidence: 99%

On the linear convergence of the stochastic gradient method with constant step-size

Cevher

Vũ

2018

Optim Lett

View full text Add to dashboard Cite

The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size γ (SGM-CS). In this paper, we provide a necessary condition, for the linear convergence of SGM-CS, that is weaker than SGC. Moreover, when this necessary is violated up to a additive perturbation σ, we show that both the projected stochastic gradient method using a constant step-size (PSGM-CS) and the proximal stochastic gradient method exhibit linear convergence to a noise dominated region, whose distance to the optimal solution is proportional to γσ.

show abstract

Restricted strong convexity and its applications to convergence analysis of gradient-type methods in convex optimization

Cited by 36 publications

References 15 publications

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization

A Generalization of Wirtinger Flow for Exact Interferometric Inversion

On the linear convergence of the stochastic gradient method with constant step-size

Contact Info

Product

Resources

About