Exploiting Heterogeneity in Robust Federated Best-Arm Identification

Mitra, Arindam; Hassani, Hamed; Pappas, George J.

doi:10.48550/arxiv.2109.05700

Cited by 5 publications

(11 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A closely related problem to ours is considered in [69], where the authors study the best arm identification in a federated setting and propose the "Federated Successive Elimination" (Fed-SEL) algorithm. [69] also considers the case where some of the participants in the federation are Byzantine clients, and it provides the robust version of Fed-SEL. [70] studies a multi-agent MAB problem where the agents collaborate to minimize their cumulative regrets, with a subset of agents being malicious.…”

Section: E Byzantine Resilient Fmabmentioning

confidence: 99%

“…[70] studies a multi-agent MAB problem where the agents collaborate to minimize their cumulative regrets, with a subset of agents being malicious. However, both [69] and [70] assume that the individual arm outcomes are i.i.d. across clients, that is, they do not consider the possible model heterogeneity arising from the client sampling discussed in Section II-D.…”

Section: E Byzantine Resilient Fmabmentioning

confidence: 99%

See 1 more Smart Citation

Federated Multi-Armed Bandits Under Byzantine Attacks

Ilker¹,

Yildirim²,

Tekin³

2022

Preprint

View full text Add to dashboard Cite

Multi-armed bandits (MAB) is a simple reinforcement learning model where the learner controls the tradeoff between exploration versus exploitation to maximize its cumulative reward. Federated multi-armed bandits (FMAB) is a recently emerging framework where a cohort of learners with heterogeneous local models play a MAB game and communicate their aggregated feedback to a parameter server to learn the global feedback model. Federated learning models are vulnerable to adversarial attacks such as model-update attacks or data poisoning. In this work, we study an FMAB problem in the presence of Byzantine clients who can send false model updates that pose a threat to the learning process. We borrow tools from robust statistics and propose a median-of-means-based estimator: Fed-MoM-UCB, to cope with the Byzantine clients. We show that if the Byzantine clients constitute at most half the cohort, it is possible to incur a cumulative regret on the order of O(log T ) with respect to an unavoidable error margin, including the communication cost between the clients and the parameter server. We analyze the interplay between the algorithm parameters, unavoidable error margin, regret, communication cost, and the arms' suboptimality gaps. We demonstrate Fed-MoM-UCB's effectiveness against the baselines in the presence of Byzantine attacks via experiments.

show abstract

Section: E Byzantine Resilient Fmabmentioning

confidence: 99%

Section: E Byzantine Resilient Fmabmentioning

confidence: 99%

Federated Multi-Armed Bandits Under Byzantine Attacks

Ilker¹,

Yildirim²,

Tekin³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [6], the honest agents face a non-stochastic (i.e., adversarial) bandit [4] and communicate at every time step, in contrast to the stochastic bandit and limited communication of our work. The authors of [47] consider the objective of best arm identification [2] instead of cumulative regret. Most of their paper involves a different communication model where the agents/clients collaborate via a central server; Section 6 studies a "peer-to-peer" model which is closer to ours but requires additional assumptions on the number of malicious neighbors.…”

Section: Other Related Workmentioning

confidence: 99%

Robust Multi-Agent Bandits Over Undirected Graphs

Vial¹,

Shakkottai²,

Srikant³

2022

Preprint

View full text Add to dashboard Cite

We consider a multi-agent multi-armed bandit setting in which n honest agents collaborate over a network to minimize regret but m malicious agents can disrupt learning arbitrarily. Assuming the network is the complete graph, existing algorithms incur O((m + K/n) log(T )/∆) regret in this setting, where K is the number of arms and ∆ is the arm gap. For m K, this improves over the single-agent baseline regret of O(K log(T )/∆).In this work, we show the situation is murkier beyond the case of a complete graph. In particular, we prove that if the state-of-the-art algorithm is used on the undirected line graph, honest agents can suffer (nearly) linear regret until time is doubly exponential in K and n. In light of this negative result, we propose a new algorithm for which the i-th agent has regret O((d mal (i) + K/n) log(T )/∆) on any connected and undirected graph, where d mal (i) is the number of i's neighbors who are malicious. Thus, we generalize existing regret bounds beyond the complete graph (where d mal (i) = m), and show the effect of malicious agents is entirely local (in the sense that only the d mal (i) malicious agents directly connected to i affect its long-term regret).

show abstract

“…The main focus in these papers is the design of coordination protocols among the agents that balance communication-efficiency with performance. A few very recent works [56][57][58][59] also look at the effect of attacks, but for the simpler unstructured multi-armed bandit problem [60]. Accounting for adversarial agents in the structured linear bandit setting we consider here requires significantly different ideas that we develop in this paper.…”

Section: Related Workmentioning

confidence: 99%

Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds

Mitra¹,

Adibi²,

Pappas³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We consider a linear stochastic bandit problem involving M agents that can collaborate via a central server to minimize regret. A fraction α of these agents are adversarial and can act arbitrarily, leading to the following tension: while collaboration can potentially reduce regret, it can also disrupt the process of learning due to adversaries. In this work, we provide a fundamental understanding of this tension by designing new algorithms that balance the exploration-exploitation trade-off via carefully constructed robust confidence intervals. We also complement our algorithms with tight analyses. First, we develop a robust collaborative phased elimination algorithm that achieves Õ α + 1/ √ M √ dT regret for each good agent; here, d is the model-dimension and T is the horizon. For small α, our result thus reveals a clear benefit of collaboration despite adversaries. Using an information-theoretic argument, we then prove a matching lower bound, thereby providing the first set of tight, near-optimal regret bounds for collaborative linear bandits with adversaries. Furthermore, by leveraging recent advances in high-dimensional robust statistics, we significantly extend our algorithmic ideas and results to (i) the generalized linear bandit model that allows for non-linear observation maps; and (ii) the contextual bandit setting that allows for time-varying feature vectors.

show abstract

Exploiting Heterogeneity in Robust Federated Best-Arm Identification

Cited by 5 publications

References 60 publications

Federated Multi-Armed Bandits Under Byzantine Attacks

Federated Multi-Armed Bandits Under Byzantine Attacks

Robust Multi-Agent Bandits Over Undirected Graphs

Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds

Contact Info

Product

Resources

About