Overlapping Multi-Bandit Best Arm Identification

Abstract

In the multi-armed bandit literature, the multi-bandit best-arm identification problem consists of determining each best arm in a number of disjoint groups of arms, with as few total arm pulls as possible. In this paper, we introduce a variant of the multi-bandit problem with overlapping groups, and present two algorithms for this problem based on successive elimination and lower/upper confidence bounds (LUCB). We bound the number of total arm pulls required for high-probability best-arm identification in every group, and we complement these bounds with a near-matching algorithm-independent lower bound. In addition, we show that a specific choice of the groups recovers the top-k ranking problem. I. INTRODUCTION The multi-armed bandit (MAB) problem [1] provides a versatile framework for sequentially searching for high-reward actions, with applications including clinical trials [2], online advertising [3], adaptive routing [4], and portfolio design [5]. A variation of the MAB problem known as multi-bandit best-arm identification consists of finding the best arm in each of a number of separate groups of arms, while pulling the minimal total number of arms possible [6]. As a motivating example, consider a scenario where each arm corresponds to a product, and pulling an arm corresponds to testing how much it is liked by some user(s). Then the multi-bandit problem corresponds to searching for the top products among multiple separate types (e.g., TV, phone, music player, etc.). Consider a variation of this example in which we not only want to find the top product of each type, but also the top products among several overlapping categories, e.g., top product under $100, top product from each brand name, top newly-released product, and so on. This motivates the overlapping multi-bandit best arm identification problem (or overlapping multi-bandit problem for short), which we introduce and study in this paper. In a nutshell, we seek to find each best arm in a number of overlapping groups using as few total arm pulls as possible; see Section II for a formal description. Beyond the preceding example, the consideration of overlapping groups is of considerable interest when arms correspond to users, since categories such as gender, age, marital status, etc. invariably exhibit overlap.

Max-Min Grouped Bandits

“…A notable grouped best-arm identification problem was studied in (Gabillon et al 2011;Bubeck, Wang, and Viswanathan 2013), where the arms are partitioned into disjoint groups, and the goal is to find the best arm in each group. A generalization of this problem to the case of overlapping groups was provided in (Scarlett, Bogunovic, and Cevher 2019). Another notable setting in which multiple arms are returned is that of subset selection, where one seeks to find a subset of k arms attaining the highest mean rewards (Kalyanakrishnan et al 2012;Kaufmann and Kalyanakrishnan 2013;Kaufmann, Cappé, and Garivier 2016).…”

“…In addition, the results of (Jedor, Perchet, and Louedec 2019) are based on the arm means satisfying certain partial ordering assumptions between the groups (e.g., all arms in a better group beat all arms in a worse group), whereas we consider general instances without such restrictions. See also (Bouneffouf et al 2019;Ban and He 2021;Singh et al 2020) and the references therein for other MAB settings with a clustering structure.…”

“…The most basic form of best-arm identification seeks to identify the arm with the highest mean (e.g., see (Kaufmann, Cappé, and Garivier 2016)). Other variations are instead based on returning multiple arms, such as the k believed to have the highest k means (Kalyanakrishnan et al 2012), or the individual highest-mean arms within a pre-defined set of groups that may be non-overlapping (Gabillon et al 2011) or overlapping (Scarlett, Bogunovic, and Cevher 2019).…”

Max-Min Grouped Bandits

“…To achieve this, we set µ 1 j " µ j `p1 `αqpµ jworstpG ˚q ´µj q for arbitrarily small α ą 0. For any arms in G with mean exactly µ jworstpG ˚q , we can perform an arbitrarily small perturbation similar to Appendix A of (Scarlett, Bogunovic, and Cevher 2019). As a result, G 1 is no longer the best group in the new instance.…”

Sequential Multi-hypothesis Testing in Multi-armed Bandit Problems:An Approach for Asymptotic Optimality

“…A special case of this problem was studied in [21], where η is the mean parameter and the distributions are sub-Gaussian, i.e., E[e sX ] ≤ e σ 2 s 2 2 , ∀s ∈ R with σ ≤ 1/2.…”