Trust-Region Methods for the Derivative-Free Optimization of Nonsmooth Black-Box Functions

Abstract

In this paper we study the minimization of a nonsmooth black-box type function, without assuming any access to derivatives or generalized derivatives and without any knowledge about the analytical origin of the function nonsmoothness. Directional methods have been derived for such problems but to our knowledge no model-based method like a trust-region one has yet been proposed. Our main contribution is thus the derivation of derivative-free trust-region methods for black-box type functions. We propose a trust-region model that is the sum of a max-linear term with a quadratic one so that the function nonsmoothness can be properly captured, but at the same time the curvature of the function in smooth subdomains is not neglected. Our trust-region methods enjoy global convergence properties similar to the ones of the directional methods, provided the vectors randomly generated for the max-linear term are asymptotically dense in the unit sphere. The numerical results reported demonstrate that our approach is both efficient and robust for a large class of nonsmooth unconstrained optimization problems. Our software is made available under request.

A weak tail-bound probabilistic condition for function estimation in stochastic derivative-free optimization

“…Indeed, in successful iterations a better control on the amplitude of the step must be exercised in order to prevent errors to accumulate too much. This allows to prove convergence of the squared trust-region radii series (i.e., the analogous of Lemma 2.1 in [15]), stating convergence to zero of the trust-region radius. Using this property, we manage to show that the proposed modification of the original Basic DFO-TRNS algorithm enjoys almost surely convergence to Clarke stationary points.…”

“…After having analyzed a simple stochastic direct-search method, we focus on a stochastic version of the Basic DFO-TRNS, presented in [15], and analyze its convergence properties under tail-bound probabilistic conditions like the ones used in Section 3. The main difference between this new version and the original Basic DFO-TRNS algorithm is in the presence of an upper bound on the trust-region radius.…”

“…In order for the method to convergence to Clarke stationary points, the sequence {g k } must be dense in the unit sphere on certain subsequences (see Theorem 3.3). We remark that a dense sequence on the unit sphere can be generated using a suitable pseudorandom sequence (see, e.g., [11,15]).…”

Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions

“…See the book [1] for an excellent treatment of this subject. Bundle methods, as well as trustregion bundle methods, have also been considered in the derivative-free setting; see, for instance, [31,42].…”

A derivative-free approach to optimal control problems with piecewise constant Mayer cost function

“…DFBBO relies only on a proximal analysis of candidate solutions close to the current incumbent, and requires neither existence nor values of gradients [6,15]. In this paper we focus on the Mads algorithm [5] implemented in the Nomad solver [2], but other techniques exist (see, e.g., [3,8,16,24]). Mads can be applied to a full discretization of Problem (P Motiv ), even if the discontinuities of the original problem are preserved.…”