Stochastic online optimization. Single-point and multi-point non-linear multi-armed bandits. Convex and strongly-convex case

Derivative-free optimization methods

“…Method types include random search (RS), mirror descent (MD) and accelerated mirror descent (AMD). MD [Gasnikov et al, 2017] squares, sparse, composite and minimax-based functional forms) and methods designed to exploit this additional structure. Although problems in this section could be solved by the general-purpose methods discussed in Sections 2 and 3, practical gains should be expected by exploiting the additional structure.…”

“…Method types include random search (RS), mirror descent (MD) and ellipsoidal. Flaxman et al, 2005] n 13/2 −2 ellipsoidal [Belloni et al, 2015] c-strongly convex, f ∈ LC 0 n 2 −3 RS [Flaxman et al, 2005] convex, f ∈ LC 1 n −3 MD [Gasnikov et al, 2017] n 13/2 −2 ellipsoidal [Belloni et al, 2015] c-strongly convex, f ∈ LC 1 n 2 −2 MD [Gasnikov et al, 2017] convex, β-smooth n 2 −2β/(β−1) RS [Bach and Perchet, 2016] c-strongly convex, β-smooth n 2 −(β+1)/(β−1) RS [Bach and Perchet, 2016] One example of a method using one-point bandit feedback, whose development falls naturally into our discussion thus far, is given by Flaxman et al [2005]; they analyse a method resembling Algorithm 6, but the gradient-free oracle is chosen as…”

“…That is, given a realization ξ ∈ Ξ, a stochastic gradient estimator based on a stochastic evaluation at the point x + µu is computed via Dekel et al [2015] and Gasnikov et al [2017]. To the best of our knowledge, the best known upper bound on the WCC for smooth, strongly convex problems is in O(n 2 −2 ) and the best known upper bound on the WCC for smooth, convex problems is in O(n −3 ) [Gasnikov et al, 2017]. For the solution of (STOCH) when Ω = R n , Bach and Perchet [2016] analyse a method resembling Algorithm 6 wherein the gradient-free oracle is chosen as…”

“…The Bregman divergence is defined by a function ψ : Ω → R, which is assumed to be 1-strongly convex with respect to the norm · p . To summarize many findings in this area [Duchi et al, 2015, Gasnikov et al, 2017, Shamir, 2017, if · q is the dual norm to · p (i.e. p −1 +q −1 = 1) where p ∈ {1, 2} and R p denotes the radius of the feasible set in the · p -norm, then a bound on WCC of type (46) in O(n 2/q R 2 p −2 ) can be established for a method like Algorithm 7 in the two-point feedback setting where f ∈ LC 0 .…”

“…Additionally assuming f (x) is strongly convex (but still assuming f ∈ LC 0 ), Agarwal et al [2010] prove, for a method like Algorithm 6, a WCC in O(n 2 −1 ). Using a method like Algorithm 7, Gasnikov et al [2017] improve the dependence on n in this WCC to O(n 2/q c −1 p −1 ), provided f is c p -strongly convex with respect to · p .…”