The computational power of optimization in online learning

Abstract

We consider the fundamental problem of prediction with expert advice where the experts are "optimizable": there is a black-box optimization oracle that can be used to compute, in constant time, the leading expert in retrospect at any point in time. In this setting, we give a novel online algorithm that attains vanishing regret with respect to N experts in total O( √ N ) computation time. We also give a lower bound showing that this running time cannot be improved (up to log factors) in the oracle model, thereby exhibiting a quadratic speedup as compared to the standard, oracle-free setting where the required time for vanishing regret is Θ(N ). These results demonstrate an exponential gap between the power of optimization in online learning and its power in statistical learning: in the latter, an optimization oracle-i.e., an efficient empirical risk minimizer-allows to learn a finite hypothesis class of size N in time O(log N ).We also study the implications of our results to learning in repeated zero-sum games, in a setting where the players have access to oracles that compute, in constant time, their best-response to any mixed strategy of their opponent. We show that the runtime required for approximating the minimax value of the game in this setting is Θ( √ N ), yielding again a quadratic improvement upon the oracle-free setting, where Θ(N ) is known to be tight.

How to Use Heuristics for Differential Privacy

“…We show that any private pERM can be efficiently used as a no-regret learning algorithm with regret guarantees that depend on the scale of the perturbations it uses. This allows us to reduce to a lower bound on the running time of oracle-efficient online learning algorithms due to Hazan and Koren [HK16]. The result is that there exist finite classes of queries Q such that any oracle-efficient differentially private pERM algorithm must introduce perturbations that are polynomially large in the size of |Q|, whereas any such class is information-theoretically privately learnable with error that scales only with log |Q|.…”

“…We wish to exploit a lower bound on the running time of oracle efficient online learners over arbitrary sets Q due to Hazan and Koren [HK16]:…”

“…Via a min-max argument together with an equilibrium sparsification technique, we can give a version of the lower bound of [HK16] that has the order of quantifiers we want -see Appendix D for the proof.…”

“…But it could be fed into our PRSMA algorithm to obtain robust privacy. It also does not solve synthetic data generation oracle efficiently because the quality score used for synthetic data generation in[BLR13] is not computable by a polynomially sized circuit generally.6 This does not contradict the lower bound of[HK16] for oracle efficient online learning, or our barrier re-…”

Oracle-Efficient Online Learning and Auction Design

“…As we show, when certain structural properties hold, the augmented history is of polynomial size even when the learner's action space is exponential, yielding oracle-efficient learning. Our results make significant progress on an open problem raised by Hazan and Koren [27], who showed that oracle-efficient algorithms do not exist in general [26] and asked whether one can identify properties under which oracle-efficient online learning may be possible.Our second main contribution is the introduction of a new adversarial online auction-design framework for revenue maximization and the application of our oracle-efficient learning results to the adaptive design of auctions. In our framework, a seller repeatedly sells an item or set of items to a population of buyers by adaptively selecting auctions from a fixed target class.…”