Inexact Tensor Methods and Their Application to Stochastic Convex Optimization

Artem, Agafonov,; Kamzolov, Dmitry; Dvurechensky, Pavel; Gasnikov, Alexander; Takáč, Martin

doi:10.48550/arxiv.2012.15636

Cited by 5 publications

(20 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…and η = e, ∇ x f (x, η) = ∇f γ (x, e), where ε > 0 is the desired accuracy for solving problem (1) in terms of the suboptimality expectation. According to Theorem 2.1, an (ε/2)-solution to ( 6) is an ε-solution to the initial problem (1). According to Theorem 2.1 and (7) we have…”

Section: Smoothing Schemementioning

confidence: 94%

“…Using these objects, we make the following assumptions. 1 • The set Q is convex and the function f is convex on the set Q γ ; 2 • The function f is Lipschitz-continuous with constant M , i.e. |f (y) − f (x)| ≤ M y − x p on Q γ , where p ∈ [1, 2] and • p is the p-norm.…”

Section: Problem Formulationmentioning

confidence: 99%

“…The simplest and most illustrative result obtained in this work is as follows. Our generic approach allows to solve problem (1) in the Euclidean geometry in O(d 1/4 M 2 R/ε) successive iterations, each requiring O(d 3/4 M 2 R/ε) oracle calls per iteration that can be made in parallel to make the total working time smaller. The dependence ∼ d 1/4 /ε corresponds to the first part of the lower bound for iteration complexity [14,8] min d 1/4 ε −1 , d 1/3 ε −2/3 .…”

Section: Problem Formulationmentioning

confidence: 99%

“…Unlike their exact first-order oracle setting, we consider zeroth-order oracle model and construct stochastic approximation to the gradient of the smoothed function. Finally, the authors of [8] propose a close technique with an accelerated higher-order method [51,27,1] playing the role of A(L, σ 2 ). For the particular setting of p = 2, they obtain a better in some regimes bound N ∼ d 1/3 /ε 2/3 for the number of iterations.…”

Section: Smoothing Schemementioning

confidence: 99%

See 3 more Smart Citations

The Power of First-Order Smooth Optimization for Black-Box Non-Smooth Problems

Gasnikov¹,

Novitskii²,

Novitskii³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Gradient-free/zeroth-order methods for black-box convex optimization have been extensively studied in the last decade with the main focus on oracle calls complexity. In this paper, besides the oracle complexity, we focus also on iteration complexity, and propose a generic approach that, based on optimal first-order methods, allows to obtain in a black-box fashion new zeroth-order algorithms for non-smooth convex optimization problems. Our approach not only leads to optimal oracle complexity, but also allows to obtain iteration complexity similar to first-order methods, which, in turn, allows to exploit parallel computations to accelerate the convergence of our algorithms. We also elaborate on extensions for stochastic optimization problems, saddle-point problems, and distributed optimization.1 Note that, for most of the algorithms in this paper, we can make these assumptions only on the intersection of Qγ and the ball x 0 + B d p (R) for some p ∈ [1, 2], where x 0 is the starting point of the algorithm and R = O x 0 − x * p ln d with x * being a solution of (1) closest to x 0 [32].

show abstract

Section: Smoothing Schemementioning

confidence: 94%

Section: Problem Formulationmentioning

confidence: 99%

Section: Problem Formulationmentioning

confidence: 99%

Section: Smoothing Schemementioning

confidence: 99%

See 2 more Smart Citations

The Power of First-Order Smooth Optimization for Black-Box Non-Smooth Problems

Gasnikov¹,

Novitskii²,

Novitskii³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, high-order methods are reasonable from the theoretical point of view for large-scale convex problems that require high accuracy. Further advantage can be potentially achieved by using inexact tensor methods (Nesterov, 2020b;Doikov & Nesterov, 2020;Agafonov et al, 2020; to save some computational work.…”

Section: A Case For High-order Methodsmentioning

confidence: 99%

Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimization

Dvurechensky¹,

Kamzolov²,

Aleksandr³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Statistical preconditioning can be used to design fast methods for distributed large-scale empirical risk minimization problems, for strongly convex and smooth loss functions, allowing fewer communication rounds. Multiple worker nodes compute gradients in parallel, which are then used by the central node to update the parameter by solving an auxiliary (preconditioned) smallerscale optimization problem. However, previous works require an exact solution of an auxiliary optimization problem by the central node at every iteration, which may be impractical. This paper proposes a method that allows the inexact solution of the auxiliary problem, reducing the total computation time. Moreover, for loss functions with high-order smoothness, we exploit the structure of the auxiliary problem and propose a hyperfast second-order method with complexity Õ(κ 1/5 ), where κ is the local condition number.

show abstract

Modeling the Convective Noise in an Electrochemical Motion Transducer

Agafonov¹

2018

International Journal of Electrochemical Science

View full text Add to dashboard Cite

This paper studies the convective noise generated in transducers found in electrochemical motion sensors. The theoretical model uses a Langevin method applied to equations of hydrodynamics. The proposed approach was applied to a transducer with a planar geometry composed of four small electrodes deposited on the walls of the thin channel of the transductive element. For realistic parameters, the modelling results agree well with the experimental data.

show abstract

Inexact Tensor Methods and Their Application to Stochastic Convex Optimization

Cited by 5 publications

References 53 publications

The Power of First-Order Smooth Optimization for Black-Box Non-Smooth Problems

The Power of First-Order Smooth Optimization for Black-Box Non-Smooth Problems

Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimization

Modeling the Convective Noise in an Electrochemical Motion Transducer

Contact Info

Product

Resources

About