Optimal Dynamic Regret in Exp-Concave Online Learning

Abstract

We consider the problem of the Zinkevich (2003)-style dynamic regret minimization in online learning with exp-concave losses. We show that whenever improper learning is allowed, a Strongly Adaptive online learner achieves the dynamic regret of Õ(dwhere Cn is the total variation (a.k.a. path length) of the an arbitrary sequence of comparators that may not be known to the learner ahead of time. Achieving this rate was highly nontrivial even for squared losses in 1D where the best known upper bound was O( √ nCn ∨ log n) (Yuan and Lamperski, 2019). Our new proof techniques make elegant use of the intricate structures of the primal and dual variables imposed by the KKT conditions and could be of independent interest. Finally, we apply our results to the classical statistical problem of locally adaptive non-parametric regression (Mammen, 1991;Donoho and Johnstone, 1998) and obtain a stronger and more flexible algorithm that do not require any statistical assumptions or any hyperparameter tuning.

Efficient and Optimal Algorithms for Contextual Dueling Bandits under Realizability

“…In such cases, one can achieve Eqn. (4) with non-trivial Reg Sq (T ) [Raj et al, 2020;Baby and Wang, 2021], which can then be used in Theorem 3.…”