Two decades of blackbox optimization applications

Alarie, Stéphane; Audet, Charles; Gheribi, Aïmen E.; Kokkolaras, Michael; Digabel, Sébastien Le

doi:10.1016/j.ejco.2021.100011

Cited by 58 publications

(24 citation statements)

References 108 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Proof. Consider the low-dimensional functions r : R p → R m and f : R p → R given by rk (ŝ) := r(x k + Q k ŝ) and f (ŝ) := 1 2 r(ŝ) 2 respectively. We note that r is continuously differentiable with Jacobian…”

Section: Constructing Q K -Fully Linear Modelsmentioning

confidence: 99%

“…This topic has received growing attention in recent years, and is primarily used for objectives which are blackbox (so analytic derivatives or algorithmic differentiation are not available), and expensive to evaluate or noisy (so finite differencing is impractical or inaccurate). There are many types of DFO methods, such as model-based, direct and pattern search, implicit filtering and others (see [50] for a recent survey), and these techniques have been used in a variety of applications [1].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Cartis¹,

Roberts²

2021

Preprint

View full text Add to dashboard Cite

We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iterations before a given optimality is achieved. This framework is specialized to nonlinear least-squares problems, with a model-based framework based on the Gauss-Newton method. This method achieves scalability by constructing local linear interpolation models to approximate the Jacobian, and computes new steps at each iteration in a subspace with user-determined dimension. We then describe a practical implementation of this framework, which we call DFBGN. We outline efficient techniques for selecting the interpolation points and search subspace, yielding an implementation that has a low per-iteration linear algebra cost (linear in the problem dimension) while also achieving fast objective decrease as measured by evaluations. Extensive numerical results demonstrate that DFBGN has improved scalability, yielding strong performance on large-scale nonlinear least-squares problems.

show abstract

Section: Constructing Q K -Fully Linear Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Cartis¹,

Roberts²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Another popular class of algorithms is based on the Mesh Adaptive Direct Search (MADS) [18,19,20], used in applications as reported in [21,22]. In general, the algorithm is composed of search (exploration) and poll (exploitation) phases, selecting evaluation points from a mesh emanating from the current best point, usually along the coordinate directions.…”

Section: Previous Workmentioning

confidence: 99%

SMGO-$Δ$: Balancing Caution and Reward in Global Optimization with Black-Box Constraints

Sabug¹,

Ruiz²,

Fagiano³

2022

Preprint

View full text Add to dashboard Cite

In numerous applications across all science and engineering areas, there are optimization problems where both the objective function and the constraints have no closed-form expression or are too complex to be managed analytically, that they can only be evaluated through experiments. To address such issues, we design a global optimization technique for problems with black-box objective and constraints. Assuming Lipschitz continuity of the cost and constraint functions, a Set Membership framework is adopted to build a surrogate model of the optimization program, that is used for exploitation and exploration routines. The resulting algorithm, named Set Membership Global Optimization With Black-Box Constraints (SMGO-∆), features one tunable risk parameter, which the user can intuitively adjust to trade-off safety, exploitation, and exploration. The theoretical properties of the algorithm are derived, and the optimization performance is compared with representative techniques from the literature in several benchmarks. Lastly, it is tested and compared with constrained Bayesian optimization in a case study pertaining to model predictive control tuning for a servomechanism with disturbances and plant uncertainties, * Corresponding author. The first author would like to acknowledge the support of the Department of Science and Technology-Science Education Institute (DOST-SEI) of the Philippines for his research.

show abstract

“…Due to the characteristics of the black-box functions, classical gradient-based optimization methods are not applicable in the BBO context. Direct-search methods (Jones et al 1993;Alarie et al 2021) and Model-based methods (Jiang et al 2020) are two well-known classes of techniques in derivative-free optimization. The former evaluates function values in a subset of sample points and selects points based on those values without any derivative approximation.…”

Section: Introductionmentioning

confidence: 99%

An Empirical Review of Model-based Adaptive Sampling for Global Optimization of Expensive Black-box Functions

Nezami¹,

Anahideh²

2022

Preprint

View full text Add to dashboard Cite

This paper reviews the state-of-the-art model-based adaptive sampling approaches for single-objective black-box optimization (BBO). While BBO literature includes various promising sampling techniques, there is still a lack of comprehensive investigations of the existing research across the vast scope of BBO problems. We first classify BBO problems into two categories: engineering design and algorithm design optimization and discuss their challenges. We then critically discuss and analyze the adaptive modelbased sampling techniques focusing on key acquisition functions. We elaborate on the shortcomings of the variance-based sampling techniques for engineering design problems. Moreover, we provide in-depth insights on the impact of the discretization schemes on the performance of acquisition functions. We emphasize the importance of dynamic discretization for distance-based exploration and introduce EEPA + , an improved variant of a previously proposed Pareto-based sampling technique. Our empirical analyses reveal the effectiveness of variance-based techniques for algorithm design and distance-based methods for engineering design optimization problems.

show abstract

Two decades of blackbox optimization applications

Cited by 58 publications

References 108 publications

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

SMGO-$Δ$: Balancing Caution and Reward in Global Optimization with Black-Box Constraints

An Empirical Review of Model-based Adaptive Sampling for Global Optimization of Expensive Black-box Functions

Contact Info

Product

Resources

About