Meta-strategy for Learning Tuning Parameters with Guarantees

Abstract

Online gradient methods, like the online gradient algorithm (OGA), often depend on tuning parameters that are difficult to set in practice. We consider an online meta-learning scenario, and we propose a meta-strategy to learn these parameters from past tasks. Our strategy is based on the minimization of a regret bound. It allows to learn the initialization and the step size in OGA with guarantees. We provide a regret analysis of the strategy in the case of convex losses. It suggests that, when there are parameters θ1, . . . , θT solving well tasks 1, . . . , T respectively and that are close enough one to each other, our strategy indeed improves on learning each task in isolation.

Online Meta-Learning in Adversarial Multi-Armed Bandits

“…Several recent works [12,17,21] introduce algorithms for online-within-online meta-learning with provable regret bounds. Each of these approaches consists of an inner algorithm that is applied to each episode and an outer algorithm that updates hyper-parameters of the inner algorithm.…”

“…They quantify the problem difficulty as the generalized empirical variance of the per-episode best-in-hindsight points. Meunier and Alquier [21] further generalize this idea and propose a unified way to treat all the hyper-parameters. Denevi et al [12] propose to use a regularized empirical error as a performance measure instead of the regret, and their outer algorithm updates the initialization point based on the (sub-)gradient of this measure.…”

“…The cornerstone of our meta-learning scheme is the ability to identify the best arm of each episode in hindsight. In the full information settings, such as [17,21], this is simply the point that minimizes the sum of loss functions. In the bandit setting it has to be estimated based on the observed partial losses.…”