Variational Analysis of Composite Models with Applications to Continuous Optimization

Mohammadi, Ashkan; Mordukhovich, Boris S.; Sarabi, M. Ebrahim

doi:10.1287/moor.2020.1074

Cited by 16 publications

(16 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Twice epi-differentiability has been recognized as an important concept of second-order variational analysis with numerous applications to optimization; see the aforementioned monograph by Rockafellar and Wets and the recent papers [45,46,47] developing a systematic approach to verify epidifferentiability via parabolic regularity, which is a major second-order property of sets and functions. The next proposition expresses the properties of the FBE ϕ γ in (5.7), which are needed for the superlinear convergence of our algorithms, in terms of the given data of (5.8).…”

mentioning

confidence: 99%

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Khanh¹,

Mordukhovich²,

Phat³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper proposes and justifies two new globally convergent Newton-type methods to solve unconstrained and constrained problems of nonsmooth optimization by using tools of variational analysis and generalized differentiation. Both methods are coderivative-based and employ generalized Hessians (coderivatives of subgradient mappings) associated with objective functions, which are either of class C 1,1 , or are represented in the form of convex composite optimization, where one of the terms may be extended-real-valued. The proposed globally convergent algorithms are of two types. The first one extends the damped Newton method and requires positive-definiteness of the generalized Hessians for its well-posedness and efficient performance, while the other algorithm is of the Levenberg-Marquardt type being well-defined when the generalized Hessians are merely positive-semidefinite. The obtained convergence rates for both methods are at least linear, but becomes superlinear under the so-called semismooth * property of subgradient mappings. Problems of convex composite optimization are investigated with and without the strong convexity assumption on of smooth parts of objective functions by implementing the machinery of forward-backward envelopes. Numerical experiments are conducted for a basic class of Lasso problems by providing performance comparisons of the new algorithms with some other first-order and second-order methods that are highly recognized in nonsmooth optimization.

show abstract

mentioning

confidence: 99%

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Khanh¹,

Mordukhovich²,

Phat³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…• we advance the variational analysis of compositions 𝑔 • 𝑐 for twice continuously differentiable 𝑐 and lower semicontinuous, not necessarily convex 𝑔, which is a more general setting than that discussed in [42,56].…”

Section: Related Work and Contributionsmentioning

confidence: 99%

Constrained composite optimization and augmented Lagrangian methods

et al. 2023

View full text Add to dashboard Cite

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class.

show abstract

“…The latter restrictions will also not allow one to have a unified analysis of amenable and non-amenable composite functions. Indeed, there is a comprehensive first and second-order order variational analysis on the composite problem (1.1) where g is convex and F is continuously twice differentiable; see [5,7,[26][27][28][33][34][35], we refer the reader to [23] for the latest and the strongest results in that topic.…”

Section: Related Workmentioning

confidence: 99%

“…The latter is due to the fact that norms are equivalent in finite dimensions. The metric subregularity condition (3.1) has shown itself a reasonable condition under which the first and second-order calculus rules hold for amenable setting both in finite and infinite-dimensional spaces; see [23,[25][26][27][28]. Metric subregularity constraint qualification is implied by Robinson constraint qualification.…”

Section: Preliminariesmentioning

confidence: 99%

“…In the following theorem, we establish a subderivative chain rule for the composite function g • F where F is semi-differentiable. A non-direction version of the latter chain rule previously obtained by author in [26] and [23] under stronger assumptions; for example, in [26,Theorem 3.4], F was assumed to be differentiable, and the (full) metric subregularity (3.1) holds. The latter would allow us to replace F by its first-order taylor expansion to reduce the problem to the problem of the chain rule for the composite function g • A, where A is an linear map.…”

Section: Calculus Rules Via Subderivativementioning

confidence: 99%

See 1 more Smart Citation

First-order variational analysis of non-amenable composite functions

Mohammadi¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper is devoted to studying the first-order variational analysis of non-convex and nondifferentiable functions that may not be subdifferentially regular. To achieve this goal, we entirely rely on two concepts of directional derivatives known as subderivative and semi-derivative. We establish the exact chain and sum rules for this class of functions via these directional derivatives. These calculus rules provide an implementable auto-differentiation process such as back-propagation in composite functions. The latter calculus rules can be used to identify the directional stationary points defined by the subderivative. We show that the distance function of a geometrically derivable constraint set is semi-differentiable, which opens the door for designing first-order algorithms for non-Clarke regular constrained optimization problems. We propose a first-order algorithm to find a directional stationary point of non-Clarke regular and perhaps non-Lipschitz functions. We introduce a descent property under which we establish the non-asymptotic convergence of our method with rate O(ε −2 ), akin to gradient descent for smooth minimization. We show that the latter descent property holds for free in some interesting non-amenable composite functions, in particular, it holds for the Moreau envelope of any bounded-below function.

show abstract

Variational Analysis of Composite Models with Applications to Continuous Optimization

Cited by 16 publications

References 32 publications

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization

Constrained composite optimization and augmented Lagrangian methods

First-order variational analysis of non-amenable composite functions

Contact Info

Product

Resources

About