A Regret-Variance Trade-Off in Online Learning

Abstract

We consider prediction with expert advice for strongly convex and bounded losses, and investigate trade-offs between regret and "variance" (i.e., squared difference of learner's predictions and best expert predictions). With K experts, the Exponentially Weighted Average (EWA) algorithm is known to achieve O(log K) regret. We prove that a variant of EWA either achieves a negative regret (i.e., the algorithm outperforms the best expert), or guarantees a O(log K) bound on both variance and regret. Building on this result, we show several examples of how variance of predictions can be exploited in learning. In the online to batch analysis, we show that a large empirical variance allows to stop the online to batch conversion early and outperform the risk of the best predictor in the class. We also recover the optimal rate of model selection aggregation when we do not consider early stopping. In online prediction with corrupted losses, we show that the effect of corruption on the regret can be compensated by a large variance. In online selective sampling, we design an algorithm that samples less when the variance is large, while guaranteeing the optimal regret bound in expectation. In online learning with abstention, we use a similar term as the variance to derive the first high-probability O(log K) regret bound in this setting. Finally, we extend our results to the setting of online linear regression.

Exponential Savings in Agnostic Active Learning Through Abstention

“…We also would like to mention that excess risk bounds can be derived from regret bounds for online learning algorithms via the onlineto-batch conversion. We refer to Gaillard and Wintenberger, 2018;Van der Hoeven et al, 2022) for related high probability upper bounds. However, in our context, the online-to-batch approach yields risk bounds with additional logarithmic factors, which makes it unsuitable for our purposes.…”

Exploring the Tradeoff between Competitive Ratio and Variance in Online-Matching Markets

“…In that context, a common practice is to formulate the problem as one of the renowned Multi-armed bandit variants [40] and then conduct regret analysis, showing the expected total regret (defined as the gap in the total utility achieved by a given policy and a prophet optimal) is upper bounded by a certain function of the total time horizon [41][42][43][44]. A few recent works investigate the potential tradeoff between variance and regret in online learning; see, e.g., [45,46]. In particular, Vakili et al [46] introduced and analyzed the performance of several risk-averse policies in both bandit and full information settings under the metric of mean-variance [47].…”