On Online Optimization: Dynamic Regret Analysis of Strongly Convex and Smooth Problems

Abstract

The regret bound of dynamic online learning algorithms is often expressed in terms of the variation in the function sequence (V_T) and/or the path-length of the minimizer sequence after T rounds. For strongly convex and smooth functions, Zhang et al. (2017) establish the squared path-length of the minimizer sequence (C*_{2,T}) as a lower bound on regret. They also show that online gradient descent (OGD) achieves this lower bound using multiple gradient queries per round. In this paper, we focus on unconstrained online optimization. We first show that a preconditioned variant of OGD achieves O(min{C*_T,C*_{2,T}}) with one gradient query per round (C*_T refers to the normal path-length). We then propose online optimistic Newton (OON) method for the case when the first and second order information of the function sequence is predictable. The regret bound of OON is captured via the quartic path-length of the minimizer sequence (C*_{4,T}), which can be much smaller than C*_{2,T}. We finally show that by using multiple gradients for OGD, we can achieve an upper bound of O(min{C*_{2,T},V_T}) on regret.

Adaptive Composite Online Optimization: Predictions in Static and Dynamic Environments

“…However, the proposed approach yields regret bounds that lack worst-case guarantees. Chang and Shahrampour [47] proposed an online optimistic Newton method that exploits gradient and hessian predictions and prove dynamic regret bounds for this algorithm.…”

Gradient-Variation Bound for Online Convex Optimization with Constraints

“…Another line of work on OCO problems considered a different regret metric, namely the dynamic regret, which is distinct from the definition in (1) (Zinkevich 2003b;Hall and Willett 2013;Yang et al 2016;Zhao et al 2020;Zhang, Lu, and Zhou 2018;Zhang et al 2017Baby and Wang 2022;Cheng et al 2020…”

“…;Chang and Shahrampour 2021;Hazan and Seshadhri 2007;Daniely, Gonen, and Shalev-Shwartz 2015;Besbes, Gur, and Zeevi 2015;Jadbabaie et al 2015;Baby, Zhao, and Wang 2021;Zhao and Zhang 2021;Goel and Wierman 2019;Mokhtari et al 2016;Zhao, Wang, and Zhou 2022;Chen, Wang, and Wang 2019;Yi et al 2021) . The dynamic regret is defined as Regret(T ) :=…”