About the Power Law of the PageRank Vector Component Distribution. Part 2. The Buckley–Osthus Model, Verification of the Power Law for This Model, and Setup of Real Search Engines

Gasnikov, A. V.; Dvurechensky, Pavel; Zhukovskii, Maksim; Kim, S. V.; Plaunov, S. S.; Smirnov, D. A.; Noskov, Fedor

doi:10.1134/s1995423918010032

Cited by 8 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach was further generalized in [30,78,90] for the case of optimization with inexact oracle for the function f . Definition 1 We say that a function f (x) is equipped with an inexact first-order oracle on a set X if there exists δ u > 0 and at any point x ∈ X for any number δ c > 0 there exists a constant L(δ c ) ∈ (0, +∞) and one can calculate f (x, δ c , δ u ) ∈ R and g(x, δ c , δ u ) ∈ R n satisfying…”

Section: Incorporating Simple Constraintsmentioning

confidence: 99%

Recent Theoretical Advances in Non-Convex Optimization

Danilova¹,

Dvurechensky²,

Gasnikov³

et al. 2020

Preprint

View full text Add to dashboard Cite

Motivated by recent increased interest in optimization algorithms for nonconvex optimization in application to training deep neural networks and other optimization problems in data analysis, we give an overview of recent theoretical results on global performance guarantees of optimization algorithms for non-convex optimization. We start with classical arguments showing that general non-convex problems could not be solved efficiently in a reasonable time. Then we give a list of problems that can be solved efficiently to find the global minimizer by exploiting the structure of the problem as much as it is possible. Another way to deal with non-convexity is to relax the goal from finding the global minimum to finding a stationary point or a local minimum. For this setting, we first present known results for the convergence rates of deterministic first-order methods, which are then followed by a general theoretical analysis of optimal stochastic and randomized gradient schemes, and an overview of the stochastic first-order methods. After that, we discuss quite general classes of non-convex problems, such as minimization of αweakly-quasi-convex functions and functions that satisfy Polyak-Łojasiewicz condition, which still allow obtaining theoretical convergence guarantees of first-order methods. Then we consider higher-order and zeroth-order/derivative-free methods and their convergence rates for non-convex optimization problems.

show abstract

Section: Incorporating Simple Constraintsmentioning

confidence: 99%

Recent Theoretical Advances in Non-Convex Optimization

Danilova¹,

Dvurechensky²,

Gasnikov³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…In the unconstrained setting, when the gradient is Lipschitz continuous, the standard gradient descent [52] achieves the lower iteration complexity bound O(ε −2 ) [19,20] to find a first-order ε-stationary point x such that ∇f (x) 2 ε. In the composite optimization setting which includes problems with simple, projectionfriendly, constraints similar iteration complexity is achieved by the mirror descent algorithm [14,34,36,48]. Various acceleration strategies of mirror and gradient descent methods have been derived in the literature, attaining the same bound as of gradient descent in the unconstrained case [35,39,40,53] or improving upon it under additional assumptions [2,18].…”

mentioning

confidence: 99%

Hessian barrier algorithms for non-convex conic optimization

Dvurechensky¹,

Staudigl²

2021

Preprint

View full text Add to dashboard Cite

We consider the minimization of a continuous function over the intersection of a regular cone with an affine set via a new class of adaptive first-and second-order optimization methods, building on the Hessian-barrier techniques introduced in Bomze et al. [16]. Our approach is based on a potential-reduction mechanism and attains a suitably defined class of approximate first-or second-order KKT points with the optimal worst-case iteration complexity O(ε −2 ) (first-order) and O(ε −3/2 ) (second order), respectively. A key feature of our methodology is the use of self-concordant barrier functions to construct strictly feasible iterates via a disciplined decomposition approach and without sacrificing on the iteration complexity of the method. To the best of our knowledge, this work is the first which achieves these worst-case complexity bounds under such weak conditions for general conic constrained optimization problems.

show abstract