Lower bounds on individual sequence regret

Gofer, Eyal; Mansour, Yishay

doi:10.1007/s10994-015-5531-y

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are not aware of any lower bounds in terms of the variance V k T . Gofer and Mansour [2012] provide lower bounds that hold for any sequence, in terms of the squared loss ranges in each round, but these do not apply to methods that adaptively tune their learning rate. For quantile bounds, Koolen [2013] takes a first step by characterizing the Pareto optimal quantile bounds for 2 experts in the √ T regime.…”

Section: Discussionmentioning

confidence: 99%

Second-order Quantile Methods for Experts and Combinatorial Games

Koolen,

van Erven

2015

Preprint

View full text Add to dashboard Cite

We aim to design strategies for sequential decision making that adjust to the difficulty of the learning problem. We study this question both in the setting of prediction with expert advice, and for more general combinatorial decision tasks. We are not satisfied with just guaranteeing minimax regret rates, but we want our algorithms to perform significantly better on easy data. Two popular ways to formalize such adaptivity are second-order regret bounds and quantile bounds. The underlying notions of 'easy data', which may be paraphrased as "the learning problem has small variance" and "multiple decisions are useful", are synergetic. But even though there are sophisticated algorithms that exploit one of the two, no existing algorithm is able to adapt to both.In this paper we outline a new method for obtaining such adaptive algorithms, based on a potential function that aggregates a range of learning rates (which are essential tuning parameters). By choosing the right prior we construct efficient algorithms and show that they reap both benefits by proving the first bounds that are both second-order and incorporate quantiles.

show abstract

Section: Discussionmentioning

confidence: 99%

Second-order Quantile Methods for Experts and Combinatorial Games

Koolen,

van Erven

2015

Preprint

View full text Add to dashboard Cite

show abstract

“…For this notion, we show that, if all predictors are individually non-discriminatory with respect to equalized error rates, running separate multiplicative weights algorithms, one for each subpopulation, preserves this non-discrimination without decay in the efficiency (Theorem 3). The key property we use is that the multiplicative weights algorithm guarantees to perform not only no worse than the best predictor in hindsight but also no better; this property holds for a broader class of algorithms [GM16]. Our result applies to general loss functions beyond binary predictions and only requires predictors to satisfy the weakened assumption of being approximately non-discriminatory.…”

Section: Our Contributionmentioning

confidence: 91%

“…A crucial property we use is that multiplicative weights not only does not perform worse than the best expert; it also does not perform better. In fact, this property holds more generally by online learning algorithms with optimal regret guarantees [GM16].…”

Section: Fairness In Composition With Respect To An Alternative Metricmentioning

confidence: 93%

On preserving non-discrimination when combining expert advice

Blum,

Gunasekar,

Lykouris

et al. 2018

Preprint

View full text Add to dashboard Cite

We study the interplay between sequential decision making and avoiding discrimination against protected groups, when examples arrive online and do not follow distributional assumptions. We consider the most basic extension of classical online learning: Given a class of predictors that are individually non-discriminatory with respect to a particular metric, how can we combine them to perform as well as the best predictor, while preserving non-discrimination? Surprisingly we show that this task is unachievable for the prevalent notion of equalized odds that requires equal false negative rates and equal false positive rates across groups. On the positive side, for another notion of non-discrimination, equalized error rates, we show that running separate instances of the classical multiplicative weights algorithm for each group achieves this guarantee. Interestingly, even for this notion, we show that algorithms with stronger performance guarantees than multiplicative weights cannot preserve non-discrimination.

show abstract

“…Algorithms exist that satisfy the even stronger property that the regret is at most ρ k and at least 0. Such non-negative-regret algorithms include all linear cost Regularized Follow the Leader algorithms, which includes Randomized Weighted Majority and linear cost Online Gradient Descent [22].…”

Section: Preliminariesmentioning

confidence: 99%

Combining No-regret and Q-learning

Kash¹,

Sullins²,

Hofmann³

2019

Preprint

View full text Add to dashboard Cite

Counterfactual Regret Minimization (CFR) has found success in settings like poker which have both terminal states and perfect recall. We seek to understand how to relax these requirements. As a first step, we introduce a simple algorithm, local no-regret learning (LONR), which uses a Q-learning-like update rule to allow learning without terminal states or perfect recall. We prove its convergence for the basic case of MDPs (and limited extensions of them) and present empirical results showing that it achieves last iterate convergence in a number of settings, most notably NoSDE games, a class of Markov games specifically designed to be challenging to learn where no prior algorithm is known to achieve convergence to a stationary equilibrium even on average.

show abstract

Lower bounds on individual sequence regret

Cited by 7 publications

References 11 publications

Second-order Quantile Methods for Experts and Combinatorial Games

Second-order Quantile Methods for Experts and Combinatorial Games

On preserving non-discrimination when combining expert advice

Combining No-regret and Q-learning

Contact Info

Product

Resources

About