Mesh adaptive direct search with second directional derivative-based Hessian update

Bűrmen, Árpád; Olenšek, Jernej; Tuma, Tadej

doi:10.1007/s10589-015-9753-5

Cited by 6 publications

(23 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was commonplace for Nomad to use thousands of seconds of CPU time, with 35 instances reaching the imposed limit of 60 minutes. This is not unexpected, as the slower convergence speed of the MADS algorithm has been observed in previous literature [13,57,63]. The number of function calls is more comparable; the average was 57,857 for DFO-VU and 41,895 for Nomad.…”

Section: Benchmark Of Dfo Solverssupporting

confidence: 83%

See 1 more Smart Citation

A derivative-free 𝒱𝒰-algorithm for convex finite-max problems

Hare

Planiden

Sagastizábal

2019

Optimization Methods and Software

View full text Add to dashboard Cite

The VU-algorithm is a superlinearly convergent method for minimizing nonsmooth, convex functions. At each iteration, the algorithm works with a certain V-space and its orthogonal U -space, such that the nonsmoothness of the objective function is concentrated on its projection onto the V-space, and on the U -space the projection is smooth. This structure allows for an alternation between a Newtonlike step where the function is smooth, and a proximal-point step that is used to find iterates with promising VU-decompositions. We establish a derivative-free variant of the VU-algorithm for convex finite-max objective functions. We show global convergence and provide numerical results from a proof-of-concept implementation, which demonstrates the feasibility and practical value of the approach. We also carry out some tests using nonconvex functions and discuss the results.

show abstract

Section: Benchmark Of Dfo Solverssupporting

confidence: 83%

“…is the number of digits of accuracy achieved by the solver. We also analyze the ability of each solver in capturing the (known) exact V-dimension, by looking at the cardinality of A(x found ) as in (13), for x found the final point found by each solver, and computing v found = |A(x found )| − 1.…”

Section: Benchmark Of Bundle Solversmentioning

confidence: 99%

A derivative-free 𝒱𝒰-algorithm for convex finite-max problems

Hare

Planiden

Sagastizábal

2019

Optimization Methods and Software

View full text Add to dashboard Cite

show abstract

“…Second-order global convergence is usually less explored in direct search, even though several algorithms attempt to use second-order aspects in their framework [7,14,31]. In fact, proving second-order results for such zeroth-order methods necessitates to strengthen the aforementioned descent requirements, thus raising the question of their practical relevance.…”

Section: Introductionmentioning

confidence: 99%

A second-order globally convergent direct-search method and its worst-case complexity

2015

View full text Add to dashboard Cite

Direct-search algorithms form one of the main classes of algorithms for smooth unconstrained derivative-free optimization, due to their simplicity and their well-established convergence results. They proceed by iteratively looking for improvement along some vectors or directions. In the presence of smoothness, first-order global convergence comes from the ability of the vectors to approximate the steepest descent direction, which can be quantified by a first-order criticality (cosine) measure. The use of a set of vectors with a positive cosine measure together with the imposition of a sufficient decrease condition to accept new iterates leads to a convergence result as well as a worst-case complexity bound. In this paper, we present a second-order study of a general class of direct-search methods. We start by proving a weak second-order convergence result related to a criticality measure defined along the directions used throughout the iterations. Extensions of this result to obtain a true second-order optimality one are discussed, one possibility being a method using approximate Hessian eigenvectors as directions (which is proved to be truly secondorder globally convergent). Numerically guaranteeing such a convergence can be rather expensive to ensure, as it is indicated by the worst-case complexity analysis provided in this paper, but turns out to be appropriate for some pathological examples.

show abstract

“…Bűrmen, Olenšek and Tuma (2015) propose a variant of with a specialized model-based search step where is a strongly convex quadratic model of and ( ) are determined from linear regression models of the constraint functions. Both the search and poll steps are accepted only if they are feasible; this corresponds to the method effectively treating the constraints with an extreme-barrier approach.…”

Section: Methods For Constrained Optimizationmentioning

confidence: 99%

Derivative-free optimization methods

2019

View full text Add to dashboard Cite

Dedicated to the memory of Andrew R. Conn for his inspiring enthusiasm and his many contributions to the renaissance of derivative-free optimization methods. AbstractIn many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.

show abstract

Mesh adaptive direct search with second directional derivative-based Hessian update

Cited by 6 publications

References 23 publications

A derivative-free 𝒱𝒰-algorithm for convex finite-max problems

A derivative-free 𝒱𝒰-algorithm for convex finite-max problems

A second-order globally convergent direct-search method and its worst-case complexity

Derivative-free optimization methods

Contact Info

Product

Resources

About