Gittins' theorem under uncertainty

Abstract

We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involves studying the interaction of our uncertainty with controls which determine the filtration. We also run a simple numerical example which illustrates the interaction between the willingness to explore and uncertainty aversion of the agent when making decisions.

Asymptotic Randomised Control with applications to bandits

“…Remark 10. Unlike in [5], where a nonlinear expectation was used to encode uncertainty aversion in the problem, here it will serve to encourage random decisions and so smooth our value function.…”