Gittins’ theorem under uncertainty

Abstract

We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under independence assumption on the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involves studying the interaction of our uncertainty with controls which determine the filtration. We also run a simple numerical example which illustrates the interaction between the willingness to explore and uncertainty aversion of the agent when making decisions.

Gambling Under Unknown Probabilities as Proxy for Real World Decisions Under Uncertainty

“…Results (6,7) are qualitatively what one would expect. Order L 2 arises as the variance of the gamblers' perceived probabilities, and order 2 as the effect of the gambler's bias when the bookmaker is not biased.…”

“…which is reasonable because a bookmaker obviously prefers gamblers to have a wide range of perceived probabilities to encourage actual betting, as will be seen in (6,7).…”

“…As perhaps the closest topic to our examples, the Gittins index [29] is analogous to the Kelly criterion in that it gives an optimal strategy within a certain "multiarmed bandits" context, assuming known distributions. A recent paper [7] describes analysis of the "uncertain distributions" case within a technically sophisticated "nonlinear expectations" framework. Final remarks.…”

“…The only situation where the casino does make a choice is in setting the odds on sports results 6. This is equivalent to paying 1 − x 1 for a contract for the opponent to win 7. In US sports betting, the width x 2 − x 1 of the spread interval is called the hold or the vig or the juice or the take or the house cut[18] 8.…”