Nondifferentiable optimization via smooth approximation: General analytical approach

Kreimer, Joseph; Rubinstein, Reuven Y.

doi:10.1007/bf02060937

Cited by 28 publications

(11 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…He developed 'random search algorithms' as first order methods using the gradient of an averaged (smoothing) function computed via convolution with mollifiers defined by probability density functions. Several authors further investigated smoothing in optimization, see Kreimer and Rubinstein [26] for a summary of those and a generalization of Katkovnik's work, and the papers [10,17,22].…”

Section: Construction Of Smoothing Functionsmentioning

confidence: 99%

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

Garmanjani

Vicente

2012

IMA Journal of Numerical Analysis

View full text Add to dashboard Cite

In the context of the derivative-free optimization of a smooth objective function, it has been shown that the worst case complexity of direct-search methods is of the same order as the one of steepest descent for derivative-based optimization, more precisely that the number of iterations needed to reduce the norm of the gradient of the objective function below a certain threshold is proportional to the inverse of the threshold squared.Motivated by the lack of such a result in the non-smooth case, we propose, analyze, and test a class of smoothing direct-search methods for the unconstrained optimization of nonsmooth functions. Given a parameterized family of smoothing functions for the non-smooth objective function dependent on a smoothing parameter, this class of methods consists of applying a direct-search algorithm for a fixed value of the smoothing parameter until the step size is relatively small, after which the smoothing parameter is reduced and the process is repeated.One can show that the worst case complexity (or cost) of this procedure is roughly one order of magnitude worse than the one for direct search or steepest descent on smooth functions.The class of smoothing direct-search methods is also showed to enjoy asymptotic global convergence properties. Some preliminary numerical experiments indicates that this approach leads to better values of the objective function, pushing in some cases the optimization further, apparently without an additional cost in the number of function evaluations.

show abstract

Section: Construction Of Smoothing Functionsmentioning

confidence: 99%

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

Garmanjani

Vicente

2012

IMA Journal of Numerical Analysis

View full text Add to dashboard Cite

show abstract

“…[5,20,21]. Then we take the version of the K-W algorithm oper-ating on the smoothed functional as in [22][23] and cf. [24,25], in order to apply the algorithm to the case when the correlation function is not unimodal.…”

Section: Focusing Algorithmsmentioning

confidence: 99%

A Simple Model for On-Sensor Phase-Detection Autofocusing Algorithm

liwiński¹,

Wachel²

2013

JCC

View full text Add to dashboard Cite

A simple model of the phase-detection autofocus device based on the partially masked sensor pixels is described. The cross-correlation function of the half-images registered by the masked pixels is proposed as a focus function. It is shown that-in such setting-focusing is equivalent to searching of the cross-correlation function maximum. Application of stochastic approximation algorithms to unimodal and non-unimodal focus functions is shortly discussed.

show abstract

“…In fact, the general theory of SF methods indicate that a variety of distributions can be used to construct smoothed functionals as long as they satisfy certain conditions [Rubinstein 1981]. A number of smoothing kernels have been studied in the literature [Rubinstein 1981; Kreimer and Rubinstein 1988;Kreimer and Rubinstein 1992;Styblinski and Tang 1990].…”

Section: Introductionmentioning

confidence: 99%

Smoothed Functional Algorithms for Stochastic Optimization Using q -Gaussian Distributions

Ghoshdastidar

Dukkipati

Bhatnagar

2014

ACM Trans. Model. Comput. Simul.

View full text Add to dashboard Cite

Smoothed functional (SF) schemes for gradient estimation are known to be efficient in stochastic optimization algorithms, specially when the objective is to improve the performance of a stochastic system. However, the performance of these methods depends on several parameters, such as the choice of a suitable smoothing kernel. Different kernels have been studied in literature, which include Gaussian, Cauchy and uniform distributions among others. This paper studies a new class of kernels based on the q-Gaussian distribution, that has gained popularity in statistical physics over the last decade. Though the importance of this family of distributions is attributed to its ability to generalize the Gaussian distribution, we observe that this class encompasses almost all existing smoothing kernels. This motivates us to study SF schemes for gradient estimation using the q-Gaussian distribution. Using the derived gradient estimates, we propose two-timescale algorithms for optimization of a stochastic objective function in a constrained setting with projected gradient search approach. We prove the convergence of our algorithms to the set of stationary points of an associated ODE. We also demonstrate their performance numerically through simulations on a queuing model.

show abstract

Nondifferentiable optimization via smooth approximation: General analytical approach

Cited by 28 publications

References 25 publications

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization

A Simple Model for On-Sensor Phase-Detection Autofocusing Algorithm

Smoothed Functional Algorithms for Stochastic Optimization Using q -Gaussian Distributions

Contact Info

Product

Resources

About