Gradient Sampling Methods for Nonsmooth Optimization

Burke, James V.; Curtis, Frank E.; Lewis, Adrian S.; Overton, Michael L.; Simões, Lucas E. A.

doi:10.1007/978-3-030-34910-3_6

Cited by 61 publications

(46 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gradient sampling methods are a developing class of algorithms for general non-smooth non-convex optimization; see the recent survey by Burke et al. (2018). These methods attempt to estimate the - subdifferential at a point by evaluating a random sample of gradients in the neighbourhood of and constructing the convex hull of these gradients.…”

Section: Deterministic Methods For Deterministic Objectivesmentioning

confidence: 99%

Derivative-free optimization methods

2019

View full text Add to dashboard Cite

Dedicated to the memory of Andrew R. Conn for his inspiring enthusiasm and his many contributions to the renaissance of derivative-free optimization methods. AbstractIn many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

show abstract

Section: Deterministic Methods For Deterministic Objectivesmentioning

confidence: 99%

Derivative-free optimization methods

2019

View full text Add to dashboard Cite

show abstract

“…Central in nonsmooth optimisation are bundle methods, where a subgradient [19] is required at each iterate to construct a linear approximation to the objective function-see [43] for an introduction. A close alternative to bundle methods are gradient sampling methods (see [12] for a recent review by Burke et al), where the descent direction is determined by sampling gradients in a neighbourhood of the current iterate. Curtis and Que [21] formulated a hybrid method between the gradient sampling scheme of [20] and the well-known quasi-Newton method BFGS adapted for nonsmooth problems [49].…”

Section: Related Literature On Nonsmooth Nonconvex Optimisationmentioning

confidence: 99%

A Geometric Integration Approach to Nonsmooth, Nonconvex Optimisation

et al. 2021

View full text Add to dashboard Cite

The optimisation of nonsmooth, nonconvex functions without access to gradients is a particularly challenging problem that is frequently encountered, for example in model parameter optimisation problems. Bilevel optimisation of parameters is a standard setting in areas such as variational regularisation problems and supervised machine learning. We present efficient and robust derivative-free methods called randomised Itoh–Abe methods. These are generalisations of the Itoh–Abe discrete gradient method, a well-known scheme from geometric integration, which has previously only been considered in the smooth setting. We demonstrate that the method and its favourable energy dissipation properties are well defined in the nonsmooth setting. Furthermore, we prove that whenever the objective function is locally Lipschitz continuous, the iterates almost surely converge to a connected set of Clarke stationary points. We present an implementation of the methods, and apply it to various test problems. The numerical results indicate that the randomised Itoh–Abe methods can be superior to state-of-the-art derivative-free optimisation methods in solving nonsmooth problems while still remaining competitive in terms of efficiency.

show abstract

“…continuous on an open dense subset of R n , which makes a step forward in nonsmooth optimization field [8]. Curtis and Overton [9] further generalized this idea to the optimization problems with nonsmooth nonlinear constraints or objective function by introducing the SQP algorithm framework.…”

Section: A Sequential Quadratic Programming Algorithm Combined With Gmentioning

confidence: 99%

Available Transfer Capability Calculation Constrained with Small-Signal Stability Based on Adaptive Gradient Sampling

Zhu

Bai

et al. 2020

Complexity

View full text Add to dashboard Cite

Due to the nonsmoothness of the small-signal stability constraint, calculating the available transfer capability (ATC) limited by small-signal stability rigorously through the nonlinear programming is quite difficult. To tackle this challenge, this paper proposes a sequential quadratic programming (SQP) method combined with gradient sampling (GS) in a dual formulation. The highlighted feature is the sample size of the gradient changes dynamically in every iteration, yielding an adaptive gradient sampling (AGS) process. Thus, the computing efficiency is greatly improved owing to the decrease and the parallelization of gradient evaluation, which dominates the computing time of the whole algorithm. Simulations on an IEEE 10-machine 39-bus system and an IEEE 54-machine 118-bus system prove the effectiveness and high efficiency of the proposed method.

show abstract

Gradient Sampling Methods for Nonsmooth Optimization

Cited by 61 publications

References 54 publications

Derivative-free optimization methods

Derivative-free optimization methods

A Geometric Integration Approach to Nonsmooth, Nonconvex Optimisation

Available Transfer Capability Calculation Constrained with Small-Signal Stability Based on Adaptive Gradient Sampling

Contact Info

Product

Resources

About