Efficient Projection-Free Online Convex Optimization with Membership Oracle

Abstract

In constrained convex optimization, existing methods based on the ellipsoid or cu ing plane method do not scale well with the dimension of the ambient space. Alternative approaches such as Projected Gradient Descent only provide a computational bene t for simple convex sets such as Euclidean balls, where Euclidean projections can be performed e ciently. For other sets, the cost of the projections can be too high. To circumvent these issues, alternative methods based on the famous Frank-Wolfe algorithm have been studied and used. Such methods use a Linear Optimization Oracle at each iteration instead of Euclidean projections; the former can o en be performed e ciently. Such methods have also been extended to the online and stochastic optimization se ings. However, the Frank-Wolfe algorithm and its variants do not achieve the optimal performance, in terms of regret or rate, for general convex sets. What is more, the Linear Optimization Oracle they use can still be computationally expensive in some cases. In this paper, we move away from Frank-Wolfe style algorithms and present a new reduction that turns any algorithm A de ned on a Euclidean ball (where projections are cheap) to an algorithm on a constrained set C contained within the ball, without sacri cing the performance of the original algorithm A by much. Our reduction requires 𝑂 (𝑇 ln𝑇 ) calls to a Membership Oracle on C a er 𝑇 rounds, and no linear optimization on C is needed. Using our reduction, we recover optimal regret bounds [resp. rates], in terms of the number of iterations, in online [resp. stochastic] convex optimization. Our guarantees are also useful in the o ine convex optimization se ing when the dimension of the ambient space is large.

Exploiting the Curvature of Feasible Sets for Faster Projection-Free Online Learning

“…centrally-symmetric] sets, without loss of generality (see e.g. [FKM05,Mha21]). For the Oracle complexity, we have that by Lemma 15 and our choice of δ in Theorem 16, the instance of Alg.…”

“…As observed by [Mha21], when it comes to the computational complexity of the Oracles, projectionfree algorithms that use LO Oracles (e.g. FW-style algorithms [HK12, HM20]) and those that use Separation/Membership Oracles (e.g.…”

“…same lines, [Mha21,GK22] recently presented algorithms that achieve the optimal O( √ T ) regret for general OCO using O(1) calls per round to a Separation/Membership Oracle. Algorithms that use Separation/Membership Oracles can be viewed as complementary to those that use an LOO (see Appendix A).…”

New Projection-free Algorithms for Online Convex Optimization with Adaptive Regret Guarantees

“…We remark that aside of standard subgradient computations of the loss functions observed, and calls to either the LOO or SO, all of our algorithms require only O(n) space, and O(T ) additional runtime (over all T iterations). We acknowledge a parallel work [19], which considers mainly projection-free OCO when the feasible set is accessible through a membership oracle. The author proves that given a separation oracle, it is possible to guarantee a O( √ T ) regret for general Lipschitz convex losses.…”

“…The author proves that given a separation oracle, it is possible to guarantee a O( √ T ) regret for general Lipschitz convex losses. However, [19] does not give adaptive regret guarantees, which is our main interest here, and does not give guarantees for the bandit setting. In particular, [19], which uses substantially different techniques than ours, requires overall O(T log T ) calls to the separation oracle to guarantee O( √ T ) regret, while our result only requires O(T ) calls in order to achieve this regret bound.…”