Twice Epi-Differentiability of Extended-Real-Valued Functions with Applications in Composite Optimization

Mohammadi, Ashkan; Sarabi, M. Ebrahim

doi:10.1137/19m1300066

Cited by 23 publications

(19 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Twice epi-differentiability has been recognized as an important concept of second-order variational analysis with numerous applications to optimization; see the aforementioned monograph by Rockafellar and Wets and the recent papers [45,46,47] developing a systematic approach to verify epidifferentiability via parabolic regularity, which is a major second-order property of sets and functions. The next proposition expresses the properties of the FBE ϕ γ in (5.7), which are needed for the superlinear convergence of our algorithms, in terms of the given data of (5.8).…”

mentioning

confidence: 99%

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Khanh¹,

Mordukhovich²,

Phat³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper proposes and justifies two new globally convergent Newton-type methods to solve unconstrained and constrained problems of nonsmooth optimization by using tools of variational analysis and generalized differentiation. Both methods are coderivative-based and employ generalized Hessians (coderivatives of subgradient mappings) associated with objective functions, which are either of class C 1,1 , or are represented in the form of convex composite optimization, where one of the terms may be extended-real-valued. The proposed globally convergent algorithms are of two types. The first one extends the damped Newton method and requires positive-definiteness of the generalized Hessians for its well-posedness and efficient performance, while the other algorithm is of the Levenberg-Marquardt type being well-defined when the generalized Hessians are merely positive-semidefinite. The obtained convergence rates for both methods are at least linear, but becomes superlinear under the so-called semismooth * property of subgradient mappings. Problems of convex composite optimization are investigated with and without the strong convexity assumption on of smooth parts of objective functions by implementing the machinery of forward-backward envelopes. Numerical experiments are conducted for a basic class of Lasso problems by providing performance comparisons of the new algorithms with some other first-order and second-order methods that are highly recognized in nonsmooth optimization.

show abstract

mentioning

confidence: 99%

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Khanh¹,

Mordukhovich²,

Phat³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Twice epi-differentiability has been recognized as an important property in second-order variational analysis with numerous applications to optimization; see the aforemention monograph by Rockafellar and Wets and the recent papers [38,39,40] developing a systematic approach to verify epidifferentiability via parabolic regularity, which is a major second-order property of sets and extendedreal-valued functions.…”

Section: Example 52 (Convex Clustering Problems)mentioning

confidence: 99%

Generalized Damped Newton Algorithms in Nonsmooth Optimization via Second-Order Subdifferentials

Khanh¹,

Mordukhovich²,

Phat³

et al. 2021

Preprint

View full text Add to dashboard Cite

The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of second-order subdifferentials of nonsmooth functions with employing the machinery of second-order variational analysis and generalized differentiation. First we develop a globally superlinearly convergent damped Newton-type algorithm for the class of continuously differentiable functions with Lipschitzian gradients, which are nonsmooth of second order. Then we design such a globally convergent algorithm to solve a class of nonsmooth convex composite problems with extended-real-valued cost functions, which typically arise in machine learning and statistics. Finally, the obtained algorithmic developments and justifications are applied to solving a major class of Lasso problems with detailed numerical implementations. We present the results of numerical experiments and compare the performance of our main algorithm applied to Lasso problems with those achieved by other first-order and second-order methods.

show abstract

“…Moreover, differentiability of the proximity operator prox Λ ϕ will remain fully equivalent to assumption (E.2) with S = C(x) under an additional parabolic derivability condition. We refer to [135,Definition 13.11 and Example 13.62] and [101,102] for more details and novel results on parabolic derivability and parabolic epi-differentiability.…”

Section: Preliminaries and Basic Differentiability Propertiesmentioning

confidence: 99%

“…Let us note that a similar result for prox-regular and subdifferentially continuous problems was recently established in [41] using the subgradient graphical derivative. Further related second-order results based on parabolic epi-differentiability and parabolic regularity can be found in [102]. We also refer to [6,58,56,53,101,102] for more discussions.…”

Section: Moreover Due To (58) We Can Infermentioning

confidence: 99%

“…Further related second-order results based on parabolic epi-differentiability and parabolic regularity can be found in [102]. We also refer to [6,58,56,53,101,102] for more discussions. A possible extension of Theorem 7.8 to the fully nonconvex, prox-regular setting as in [41,Theorem 3.8] and [102, Theorem 6.1 and 6.3] is left for future work.…”

Section: Moreover Due To (58) We Can Infermentioning

confidence: 99%

See 1 more Smart Citation

A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

Ouyang,

Milzarek

2021

Preprint

View full text Add to dashboard Cite

We propose a novel trust region method for solving a class of nonsmooth and nonconvex compositetype optimization problems. The approach embeds inexact semismooth Newton steps for finding zeros of a normal map-based stationarity measure for the problem in a trust region framework. Based on a new merit function and acceptance mechanism, global convergence and transition to fast local q-superlinear convergence are established under standard conditions. In addition, we verify that the proposed trust region globalization is compatible with the Kurdyka-Łojasiewicz (KL) inequality yielding finer convergence results. We further derive new normal map-based representations of the associated second-order optimality conditions that have direct connections to the local assumptions required for fast convergence. Finally, we study the behavior of our algorithm when the Hessian matrix of the smooth part of the objective function is approximated by BFGS updates. We successfully link the KL theory, properties of the BFGS approximations, and a Dennis-Moré-type condition to show superlinear convergence of the quasi-Newton version of our method. Numerical experiments on sparse logistic regression and image compression illustrate the efficiency of the proposed algorithm.

show abstract

Twice Epi-Differentiability of Extended-Real-Valued Functions with Applications in Composite Optimization

Cited by 23 publications

References 15 publications

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Generalized Damped Newton Algorithms in Nonsmooth Optimization via Second-Order Subdifferentials

A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

Contact Info

Product

Resources

About