Optimal algorithms for global optimization in case of unknown Lipschitz constant

Horn, Matthias

doi:10.1016/j.jco.2005.06.006

Cited by 14 publications

(22 citation statements)

References 7 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main point of [5] was to study whether estimates of this type can be obtained without this knowledge. We call such algorithms universal.…”

Section: It Follows Thatmentioning

confidence: 99%

“…We call such algorithms universal. A universal algorithm was given in [5] for functions f in Lip M * 1, which does not require knowledge of M * , but still yields an estimate…”

Section: It Follows Thatmentioning

confidence: 99%

“…Notice, that since |Ω| = 1, the condition (1.4) automatically holds for δ large by adjusting L. The previous work in [8,5] assumes that (1.4) only holds for δ ≤ δ 0 which may enable the introduction of a smaller L, but then all results also involve the constant δ 0 . Our results could also be presented in that way, but at the expense of much more complicated statements which we choose not to make.…”

mentioning

confidence: 99%

“…Adaptive query algorithms for finding the minimum of a function f are studied. The algorithms build on the earlier adaptive algorithms given in [5,8]. The rate of convergence of these algorithms is estimated under various model assumptions on the function f .…”

mentioning

confidence: 99%

See 3 more Smart Citations

Finding the minimum of a function

Cohen

DeVore

Petrova

et al. 2013

Methods and Applications of Analysis

View full text Add to dashboard Cite

Abstract. Adaptive query algorithms for finding the minimum of a function f are studied. The algorithms build on the earlier adaptive algorithms given in [5,8]. The rate of convergence of these algorithms is estimated under various model assumptions on the function f . The first class of algorithms is analyzed when f satisfies a smoothness condition, e.g. f ∈ C r , and an assumption on its level sets as given in [8]. There is a distinction drawn as to whether or not the algorithm has knowledge of the semi-norm |f | C r . If this information is known, it is rather straightforward to design algorithms with optimal performance and to show that this performance is better than nonadaptive algorithms. A bit more subtle is to build algorithms which are universal in that they do not need to know the semi-norm of f . Universal algorithms are built that have the same asymptotic performance as when the semi-norm is known, save for a logarithm. The second part of this paper studies adaptive algorithms for finding the minimum of a function in high dimension. In this case, additional assumptions are placed on f , of the form given in [3], that have the effect of variable reduction and thereby avoiding the curse of dimensionality. These algorithms are again shown to be asymptotically optimal up to a logarithm factor.

show abstract

“…The main point of [5] was to study whether estimates of this type can be obtained without this knowledge. We call such algorithms universal.…”

Section: It Follows Thatmentioning

confidence: 99%

“…We call such algorithms universal. A universal algorithm was given in [5] for functions f in Lip M * 1, which does not require knowledge of M * , but still yields an estimate…”

Section: It Follows Thatmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Finding the minimum of a function

Cohen

DeVore

Petrova

et al. 2013

Methods and Applications of Analysis

View full text Add to dashboard Cite

show abstract

“…In every phase those instances are selected that may converge to the best value for a particular range of the constant. The selection mechanism is analogous to the DIRECT algorithm [15,8,14] for optimizing Lipschitz-functions with an unknown constant, where preference is given to rectangles that may contain the global optimum. The optimum within each rectangle is estimated in an optimistic way, and the estimate depends on the size of the rectangle.…”

Section: Introductionmentioning

confidence: 99%

Efficient Multi-start Strategies for Local Search Algorithms

Kocsis

György

2009

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Local search algorithms for global optimization often suffer from getting trapped in a local optimum. The common solution for this problem is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resources (in particular, processing time) to the instances depending on their behavior. Hence, a multi-start strategy has to decide (dynamically) when to allocate additional resources to a particular instance and when to start new instances. In this paper we propose a consistent multi-start strategy that assumes a convergence rate of the local search algorithm up to an unknown constant, and in every phase gives preference to those instances that could converge to the best value for a particular range of the constant. Combined with the local search algorithm SPSA (Simultaneous Perturbation Stochastic Approximation), the strategy performs remarkably well in practice, both on synthetic tasks and on tuning the parameters of learning algorithms.

show abstract

A data variability index: Quantifying complexity of models and analyzing adversarial data

Al‐Hmouz

Pedrycz

Ammari

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

In system modeling arises a fundamental question about the level of difficulty one may encounter when designing a model on a basis of some training data. In this study, we advocate that such level of difficulty inherently depends upon the variability of the available function (data). If for a pair of input data which exhibits small differences, the differences of the corresponding outputs are substantial then building a model in the presence of such data becomes more challenging than in cases of data where the differences in the output data are far more limited. Dwelling on this observation, we introduce a variability index quantifying the nature of data in terms of variability observed in input and output data, respectively. The proposed index is model-neutral (model agnostic), namely describes and quantifies the modeling challenge implied by the data irrespectively of the specific model to be constructed. In case of functions, we show that the Lipschitz constant plays a similar role as the variability index computed for experimental data. An original way of reducing values of the variability index through a nonlinear transformation of original data completed by a fuzzy rule-based model is introduced. It is shown that such rule-based architecture gives rise to a piecewise linear transformation (multipoint linear approximation) exhibiting required contraction-dilation

show abstract

Optimal algorithms for global optimization in case of unknown Lipschitz constant

Cited by 14 publications

References 7 publications

Finding the minimum of a function

Finding the minimum of a function

Efficient Multi-start Strategies for Local Search Algorithms

A data variability index: Quantifying complexity of models and analyzing adversarial data

Contact Info

Product

Resources

About