Black-Box Reductions for Parameter-free Online Learning in Banach Spaces

Abstract

We introduce several new black-box reductions that significantly improve the design of adaptive and parameterfree online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce optimization in a Banach space to one-dimensional optimization, and we reduce optimization over a constrained domain to unconstrained optimization. All of our reductions run as fast as online gradient descent. We use our new techniques to improve upon the previously best regret bounds for parameter-free learning, and do so for arbitrary norms.

Artificial Constraints and Lipschitz Hints for Unconstrained Online Learning

“…First, we observe that it suffices to achieve our desired bounds in the one-dimensional case W = R, as it is easy to convert any one-dimensional algorithm to a dimension-free algorithm via a recent reduction argument [4] (see Section A for details). Our one-dimensional algorithm is then constructed via three steps:…”

“…From the previous step, it seems that we should try to control max t |w t |. We do this by enforcing an "artificial constraint": we use the constraint set reduction in [4] to ensure |w t | ≤ √ T for all t. This results in good regret for all |ẘ| ≤ √ T , but does not control regret for |ẘ| > √ T . To address |ẘ| > √ T , we then observe that R T (ẘ) ≤ R T (0)+ |ẘ|GT ≤ R T (0)+ G|ẘ| 3 and use the fact that R T (0) is constant (because |0| ≤ √ T ) to conclude the desired results (Section 5).…”

“…To accomplish this, we adapt the algorithm of [4] to this setting and obtain an algorithm that guarantees regret…”

“…Thus the problem of choosing w t reduces to the problem of choosing v t . We solve this problem in the same way as [4] by recasting choosing v t as an online exp-concave optimization problem in 1 dimension, and then use the Online Newton Step (ONS) algorithm [8] to optimize v t .…”

“…The analysis of this algorithm is nearly identical to that in [4], although we reproduce the main steps for completeness. The major deviation is that we are performing ONS updates on shrinking domains [−1/2h t , 1/2h t ], which we analyze carefully in appendix.…”