Accelerated Gradient-Free Optimization Methods with a Non-Euclidean Proximal Operator

Vorontsova, Evgeniya; Gasnikov, Alexander; Gorbunov, Eduard; Двуреченский, П Е

doi:10.1134/s0005117919080095

Cited by 9 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It seems that one can do it analogously. 5 Generalization of the results of [5,10,17] and [1,13] for the gradient-free saddle-point set-up is more challenging. Also, based on combinations of ideas from [1,11] it'd be interesting to develop a mixed method with a gradient oracle for x (outer minimization) and a gradientfree oracle for y (inner maximization).…”

Section: Possible Generalizationsmentioning

confidence: 99%

Gradient-Free Methods with Inexact Oracle for Convex-Concave Stochastic Saddle-Point Problem

Beznosikov¹,

Sadiev²,

Gasnikov³

2020

Mathematical Optimization Theory and Operations Research

View full text Add to dashboard Cite

In the paper, we generalize the approach Gasnikov et. al, 2017, which allows to solve (stochastic) convex optimization problems with an inexact gradient-free oracle, to the convex-concave saddle-point problem. The proposed approach works, at least, like the best existing approaches. But for a special set-up (simplex type constraints and closeness of Lipschitz constants in 1 and 2 norms) our approach reduces n /log n times the required number of oracle calls (function calculations). Our method uses a stochastic approximation of the gradient via finite differences. In this case, the function must be specified not only on the optimization set itself, but in a certain neighbourhood of it. In the second part of the paper, we analyze the case when such an assumption cannot be made, we propose a general approach on how to modernize the method to solve this problem, and also we apply this approach to particular cases of some classical sets.

show abstract

Section: Possible Generalizationsmentioning

confidence: 99%

Gradient-Free Methods with Inexact Oracle for Convex-Concave Stochastic Saddle-Point Problem

Beznosikov¹,

Sadiev²,

Gasnikov³

2020

Mathematical Optimization Theory and Operations Research

View full text Add to dashboard Cite

show abstract

“…Mainly driven by applications in imaging and machine learning, the idea of acceleration turned out to be very productive in the last 20 years. During this time span it has been extended to composite optimization [54,133], general proximal setups [67,26], stochastic optimization problems [134,135,136,137,138,139,140], optimization with inexact oracle [141,142,138,139,143,144,145,57], variance reduction methods [148,149,150,151,152,153], alternating minimization methods [154,155], random coordinate descent [156,157,158,159,160,161,162,163,164,154] and other randomized methods such as randomized derivative-free methods [165,164,166,167] and randomized directional search [164,168,169],…”

Section: Accelerated Methodsmentioning

confidence: 99%

First-Order Methods for Convex Optimization

Dvurechensky¹,

Staudigl²,

Shtern³

2021

Preprint

View full text Add to dashboard Cite

First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal-dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.

show abstract

“…In [Nesterov and Spokoiny, 2017], the authors consider random derivative-free methods and provide them with some complexity bounds for different classes of convex optimization problems as well as accelerated methods for smooth convex derivative-free optimization. In [Vorontsova et al, 2019], the authors propose an accelerated gradient-free method with a non-Euclidean proximal operator. Paper [Gorbunov et al, 2022] describes an accelerated method for smooth stochastic derivative-free optimization with two-point feedback.…”

Section: Introductionmentioning

confidence: 99%

Partially observed distributed optimization under unknown-but-bounded disturbances

Erofeeva

Kizhaeva

2023

CAP

View full text Add to dashboard Cite

In this paper, we consider non-stationary distributed optimization with partially observed parameters with acceleration based on the estimate sequence proposed by Y. Nesterov. We formulate this partial observability as time-varying communication matrix defined for each parameter separately. We propose the new distributed algorithm combining the accelerated Simultaneous Perturbation Stochastic Approximation (SPSA) and the described communication scheme as well as show its theoretical properties. The simulation validates the proposed algorithm in multi-sensor multi-target tracking problem over delayed channels.

show abstract

Accelerated Gradient-Free Optimization Methods with a Non-Euclidean Proximal Operator

Cited by 9 publications

References 6 publications

Gradient-Free Methods with Inexact Oracle for Convex-Concave Stochastic Saddle-Point Problem

Gradient-Free Methods with Inexact Oracle for Convex-Concave Stochastic Saddle-Point Problem

First-Order Methods for Convex Optimization

Partially observed distributed optimization under unknown-but-bounded disturbances

Contact Info

Product

Resources

About