A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming

Borges, Pedro Augusto Pereira; Sagastizábal, Claudia; Solodov, M. V.

doi:10.1007/s10107-020-01582-2

Cited by 15 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(14) After that, using the same idea of sequential minimization method, inspired by the regularized smoothing method in [46], the value-function f * (x) can be approximated with a barrier function (different from Eq. ( 6) due to the LL constraints) and a regularization term:…”

Section: Extension For Blo With Functional Constraintsmentioning

confidence: 99%

Value-Function-based Sequential Minimization for Bi-level Optimization

Liu¹,

Li²,

Zeng³

et al. 2021

Preprint

View full text Add to dashboard Cite

Gradient-based Bi-Level Optimization (BLO) methods have been widely applied to handle modern learning tasks. However, most existing strategies are theoretically designed based on restrictive assumptions (e.g., convexity of the lower-level sub-problem), and computationally not applicable for high-dimensional tasks. Moreover, there are almost no gradient-based methods able to solve BLO in those challenging scenarios, such as BLO with functional constraints and pessimistic BLO. In this work, by reformulating BLO into approximated single-level problems, we provide a new algorithm, named Bi-level Value-Function-based Sequential Minimization (BVFSM), to address the above issues. Specifically, BVFSM constructs a series of value-function-based approximations, and thus avoids repeated calculations of recurrent gradient and Hessian inverse required by existing approaches, time-consuming especially for high-dimensional tasks. We also extend BVFSM to address BLO with additional functional constraints. More importantly, BVFSM can be used for the challenging pessimistic BLO, which has never been properly solved before. In theory, we prove the convergence of BVFSM on these types of BLO, in which the restrictive lower-level convexity assumption is completely discarded. To our best knowledge, this is the first gradient-based algorithm that can solve different kinds of BLO (e.g., optimistic, pessimistic, and with constraints) with solid convergence guarantees. Extensive experiments verify the theoretical investigations and demonstrate our superiority on various real-world applications.

show abstract

Section: Extension For Blo With Functional Constraintsmentioning

confidence: 99%

Value-Function-based Sequential Minimization for Bi-level Optimization

Liu¹,

Li²,

Zeng³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the inequality constraint f (x, y) ≤ f * (x) is illposed, in the sense that f * (x) is non-smooth and the constraint does not satisfy any standard regularity condition. To circumvent such difficulty, inspired by (Borges et al, 2020), we relax such constraint by replacing f * (x) with the value function of the regularized LL problem, i.e.,…”

Section: Bi-level Value-function-basedmentioning

confidence: 99%

A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization

Liu,

Yuan

et al. 2021

Preprint

View full text Add to dashboard Cite

Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest. Due to the witnessed efficiency in solving bi-level programs, gradient-based methods have gained popularity in the machine learning community. In this work, we propose a new gradient-based solution scheme, namely, the Bilevel Value-Function-based Interior-point Method (BVFIM). Following the main idea of the logbarrier interior-point scheme, we penalize the regularized value function of the lower level problem into the upper level objective. By further solving a sequence of differentiable unconstrained approximation problems, we consequently derive a sequential programming scheme. The numerical advantage of our scheme relies on the fact that, when gradient methods are applied to solve the approximation problem, we successfully avoid computing any expensive Hessian-vector or Jacobian-vector product. We prove the convergence without requiring any convexity assumption on either the upper level or the lower level objective. Experiments demonstrate the efficiency of the proposed BVFIM on non-convex bi-level problems.

show abstract

“…This assumption is very restrictive and cannot hold for most applications. However, inducing second order sufficient conditions for approximating problems can give rise to efficient smoothing methods for many problems involving nonsmooth functions [5,6].…”

Section: Aim Of the Workmentioning

confidence: 99%

Estimates of Generalized Hessians for Optimal Value Functions in Mathematical Programming

Zemkoho

2021

Set-Valued Var. Anal

View full text Add to dashboard Cite

We consider the optimal value function of a parametric optimization problem. A large number of publications have been dedicated to the study of continuity and differentiability properties of the function. However, the differentiability aspect of works in the current literature has mostly been limited to first order analysis, with focus on estimates of its directional derivatives and subdifferentials, given that the function is typically nonsmooth. With the progress made in the last two to three decades in major subfields of optimization such as robust, minmax, semi-infinite and bilevel optimization, and their connection to the optimal value function, there is a need for a second order analysis of the generalized differentiability properties of this function. This could enable the development of robust solution algorithms, such as the Newton method. The main goal of this paper is to provide estimates of the generalized Hessian for the optimal value function. Our results are based on two handy tools from parametric optimization, namely the optimal solution and Lagrange multiplier mappings, for which completely detailed estimates of their generalized derivatives are either well-known or can easily be obtained.

show abstract

A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming

Cited by 15 publications

References 34 publications

Value-Function-based Sequential Minimization for Bi-level Optimization

Value-Function-based Sequential Minimization for Bi-level Optimization

A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization

Estimates of Generalized Hessians for Optimal Value Functions in Mathematical Programming

Contact Info

Product

Resources

About