Collaborative Top Distribution Identifications with Limited Interaction (Extended Abstract)

Collaborative Pure Exploration in Kernel Bandit

“…Eq. ( 1) allows tasks to be different for agents, whereas prior CoPE works [20,22,38] restrict the tasks (X 𝑣 and 𝑓 𝑣 ) to be the same for all agents 𝑣 ∈ [𝑉 ].…”

“…Communication. Following the popular communication protocol in existing CoPE works [20,22,38], we allow these 𝑉 agents to exchange information via communication rounds, in which each agent can broadcast and receive messages from others. While we do not restrict the exact length of a message, for practical implementation it should be bounded by 𝑂 (𝑛) bits.…”

“…To evaluate the learning acceleration of our algorithms, following the CoPE literature, e.g., [20,22,38], we also define the speedup metric of our algorithms. For a CoPE-KB instance I, let 𝑇 A 𝑀 ,I denote the average number of samples used by each agent in multi-agent algorithm A 𝑀 to identify 𝑥 𝑣, * for all 𝑣 ∈ [𝑉 ], and let 𝑇 A 𝑆 ,I denote the average number of samples used by each task for a single-agent algorithm A 𝑆 to sequentially (without communication) identify 𝑥 𝑣, * for all 𝑣 ∈ [𝑉 ].…”

“…In such applications, it is important to develop a more general CoPE model that allows heterogeneous tasks and complex reward structures, and quantitatively investigate how task similarities impact learning acceleration. Motivated by the above facts, we propose a novel Collaborative Pure Exploration in Kernel Bandit (CoPE-KB) problem, which generalizes traditional single-task CoPE problems [20,22,38] to the multi-task setting. It also generalizes the classic MAB model to allow general (linear or nonlinear) reward structures via the powerful kernel representation.…”

“…Our work distinguishes itself from prior CoPE works, e.g., [20,22,38], in the following aspects: (i) Prior works [20,22,38] only consider the classic MAB setting, while we adopt a high-dimensional RKHS to allow more general real-world reward dependency on option features. (ii) Unlike [20,22,38] which restrict tasks (given arm sets and rewards) among agents to be the same, we allow different tasks for different agents, and explicitly quantify how task similarities impact learning acceleration. (iii) In lower bound analysis, prior works [20,38] mainly focus on a 2-armed case, whereas we derive a novel lower bound analysis for general multi-armed cases with high-dimensional linear reward structures.…”