The spectral bundle method with second-order information

Helmberg, Christoph; Overton, Michael L.; Rendl, Franz

doi:10.1080/10556788.2013.858155

Cited by 17 publications

(13 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Large scale SDPs pointed research towards first-order approaches, which are more computationally appealing. For linear f , we note among others the work of [76], a provably convergent alternating direction augmented Lagrangian algorithm, and that of Helmberg and Rendl [36], where they develop an efficient first-order spectral bundle method for SDPs with the constant trace property; see also [35] for extensions on this line of work. In both cases, no convergence rate guarantees are provided; see also [57].…”

Section: Related Workmentioning

confidence: 99%

“…In the above inequalities (35), we used the fact that symmetric version of A is a PSD matrix, where A := ∇f (X)∆(U +U r R U ) = ∇f (X)•(X−X r ) is a PSD matrix, i.e., given a vector y, y ∇f (X)•(X−X r )y ≥ 0. To show this, let g(t) = f (X + tyy ) be a function from R → R. Hence, ∇g(t) = ∇f (X + tyy ), yy .…”

Section: Main Lemmas For the Smooth Casementioning

confidence: 99%

See 1 more Smart Citation

Dropping Convexity for Faster Semi-definite Optimization

Bhojanapalli¹,

Kyrillidis²,

Sanghavi³

2015

Preprint

View full text Add to dashboard Cite

We study the minimization of a convex function f (X) over the set of n × n positive semi-definite matrices, but when the problem is recast as minU g(U ) := f (U U ), with U ∈ R n×r and r ≤ n. We study the performance of gradient descent on g-which we refer to as Factored Gradient Descent (Fgd)-under standard assumptions on the original function f .We provide a rule for selecting the step size and, with this choice, show that the local convergence rate of Fgd mirrors that of standard gradient descent on the original f : i.e., after k steps, the error is O(1/k) for smooth f , and exponentially small in k when f is (restricted) strongly convex. In addition, we provide a procedure to initialize Fgd for (restricted) strongly convex objectives and when one only has access to f via a first-order oracle; for several problem instances, such proper initialization leads to global convergence guarantees.Fgd and similar procedures are widely used in practice for problems that can be posed as matrix factorization. To the best of our knowledge, this is the first paper to provide precise convergence rate guarantees for general convex functions under standard convex assumptions.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Main Lemmas For the Smooth Casementioning

confidence: 99%

Dropping Convexity for Faster Semi-definite Optimization

Bhojanapalli¹,

Kyrillidis²,

Sanghavi³

2015

Preprint

View full text Add to dashboard Cite

show abstract

“…It repeatedly projects A (ω) to small subspaces, and minimizes the largest eigenvalue of the resulting projected matrix-valued function. Such subspace ideas have also been explored in special contexts such as convex semidefinite programs (Helmberg & Rendl (2000); Helmberg et al (2014)), and the computation of the pseudospectral abscissa (Kressner & Vandereycken (2014); Meerbergen et al (2017)).…”

Section: Introductionmentioning

confidence: 99%

Nonsmooth algorithms for minimizing the largest eigenvalue with applications to inner numerical radius

Kangal

Mengi

2019

IMA Journal of Numerical Analysis

View full text Add to dashboard Cite

Nonsmoothness at optimal points is a common phenomenon in many eigenvalue optimization problems. We consider two recent algorithms to minimize the largest eigenvalue of a Hermitian matrix dependent on one parameter, both proven to be globally convergent unaffected by nonsmoothness. One of these algorithms models the eigenvalue function with a piece-wise quadratic function and is effective in dealing with nonconvex problems. The other algorithm projects the Hermitian matrix into subspaces formed of eigenvectors and is effective in dealing with large-scale problems. We generalize the latter slightly to cope with nonsmoothness. For both algorithms we analyze the rate of convergence in the nonsmooth setting, when the largest eigenvalue is multiple at the minimizer and zero is strictly in the interior of the generalized Clarke derivative, and prove that both algorithms converge rapidly. The algorithms are applied to, and the deduced results are illustrated on the computation of the inner numerical radius, the modulus of the point on the boundary of the field of values closest to the origin, which carries significance for instance for the numerical solution of a symmetric definite generalized eigenvalue problem and the iterative solution of a saddle point linear system.

show abstract

“…section 4.1) is only one possible algorithm among many. Other more specialized methods, such as the spectral bundle method of Helmberg and Rendl (2000), its second-order variant (Helmberg, Overton, and Rendl, 2014), or the stochastic-gradient method of d'Aspremont and Karoui (2014), may prove effective alternatives.…”

Section: Blind Deconvolutionmentioning

confidence: 99%

Low-Rank Spectral Optimization via Gauge Duality

Friedlander¹,

Macêdo²

2016

SIAM J. Sci. Comput.

View full text Add to dashboard Cite

Abstract. Various applications in signal processing and machine learning give rise to highly structured spectral optimization problems characterized by low-rank solutions. Two important examples that motivate this work are optimization problems from phase retrieval and from blind deconvolution, which are designed to yield rank-1 solutions. An algorithm is described that is based on solving a certain constrained eigenvalue optimization problem that corresponds to the gauge dual which, unlike the more typical Lagrange dual, has an especially simple constraint. The dominant cost at each iteration is the computation of rightmost eigenpairs of a Hermitian operator. A range of numerical examples illustrate the scalability of the approach.

show abstract

The spectral bundle method with second-order information

Cited by 17 publications

References 30 publications

Dropping Convexity for Faster Semi-definite Optimization

Dropping Convexity for Faster Semi-definite Optimization

Nonsmooth algorithms for minimizing the largest eigenvalue with applications to inner numerical radius

Low-Rank Spectral Optimization via Gauge Duality

Contact Info

Product

Resources

About