Statistically Robust, Risk-Averse Best Arm Identification in Multi-Armed Bandits

Abstract

Traditional multi-armed bandit (MAB) formulations usually make certain assumptions about the underlying arms' distributions, such as bounds on the support or their tail behaviour. Moreover, such parametric information is usually 'baked' into the algorithms. In this paper, we show that specialized algorithms that exploit such parametric information are prone to inconsistent learning performance when the parameter is misspecified. Our key contributions are twofold: (i) We establish fundamental performance limits of statistically robust MAB algorithms under the fixedbudget pure exploration setting, and (ii) We propose two classes of algorithms that are asymptotically near-optimal. Additionally, we consider a risk-aware criterion for best arm identification, where the objective associated with each arm is a linear combination of the mean and the conditional value at risk (CVaR). Throughout, we make a very mild 'bounded moment' assumption, which lets us work with both light-tailed and heavy-tailed distributions within a unified framework.

Risk averse non-stationary multi-armed bandits

“…Upper confidence bound algorithms in this context are studied by Maillard (2013), Cassel et al (2018), Khajonchotpanya et al (2021). Alternative arm selection approaches in the context of risk-averse bandits include the max-min approach discussed in Galichet et al (2013), the successive rejects relying on concentration bound guarantees of Kolla et al (2019a), robust estimation-based algorithms in Kagrecha et al (2020), or Thompson Sampling approaches in Chang et al (2020) and Baudry et al (2021).…”