Adversarial Attacks On Multi-Agent Communication

Tu, James; Wang, Tsun-Hsuan; Wang, Jingkang; Manivasagam, Sivabalan; Ren, Mengye; Urtasun, Raquel

doi:10.1109/iccv48922.2021.00767

Cited by 25 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [33], an adversarial mesh is put on the rooftop of a vehicle, and mesh poses are optimized to make the vehicle invisible. Besides the above single-agent attack, [36] attacks the V2X perception system by adding adversarial noises to the intermediate features shared by intelligent agents. Different from previous work, we directly attack the agent collaboration choice and vehicle poses to deteriorate the performance of the modern V2X perception system, which ensures the feasibility and plausibility of the generated adversarial examples.…”

Section: Related Workmentioning

confidence: 99%

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

Xiang¹,

Xu²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent advancements in Vehicle-to-Everything communication technology have enabled autonomous vehicles to share sensory information to obtain better perception performance. With the rapid growth of autonomous vehicles and intelligent infrastructure, the V2X perception systems will soon be deployed at scale, which raises a safety-critical question: how can we evaluate and improve its performance under challenging traffic scenarios before the real-world deployment? Collecting diverse large-scale real-world test scenes seems to be the most straightforward solution, but it is expensive and timeconsuming, and the collections can only cover limited scenarios. To this end, we propose the first open adversarial scene generator V2XP-ASG that can produce realistic, challenging scenes for modern LiDAR-based multi-agent perception system. V2XP-ASG learns to construct an adversarial collaboration graph and simultaneously perturb multiple agents' poses in an adversarial and plausible manner. The experiments demonstrate that V2XP-ASG can effectively identify challenging scenes for a large range of V2X perception systems. Meanwhile, by training on the limited number of generated challenging scenes, the accuracy of V2X perception systems can be further improved by 12.3% on challenging and 4% on normal scenes.

show abstract

Section: Related Workmentioning

confidence: 99%

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

Xiang¹,

Xu²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Efficient collaboration depends on reliable communication among agents. However, communication is not perfect in practice: i) As the number of vehicles in the network increases, available communication bandwidth for each vehicle is limited; ii) It is hard for vehicles to receive real-time information from other vehicles due to inevitable communication delay; iii) Communication is likely to be interrupted sometimes, resulting in communication drops; iv) V2X communication suffers from attacks [34] and cannot always provide reliable service. Although communication technologies is keep developing and improving quality of communication service, the problems mentioned above will still exist for a long time.…”

Section: Communication Robustnessmentioning

confidence: 99%

Collaborative Perception for Autonomous Driving: Current Status and Future Trend

Ren

Chen

Zhang

2022

Proceedings of 2021 5th Chinese Conference on Swarm Intelligence and Cooperative Control

View full text Add to dashboard Cite

Perception is one of the crucial module of the autonomous driving system, which has made great progress recently. However, limited ability of individual vehicles results in the bottleneck of improvement of the perception performance. To break through the limits of individual perception, collaborative perception has been proposed which enables vehicles to share information to perceive the environments beyond line-of-sight and field-of-view. In this paper, we provide a review of the related work about the promising collaborative perception technology, including introducing the fundamental concepts, generalizing the collaboration modes and summarizing the key ingredients and applications of collaborative perception. Finally, we discuss the open challenges and issues of this research area and give some potential further directions.

show abstract

“…Communication is important for agents to obtain high rewards, but it can be a double-edged swordit benefits decision making but may make agents vulnerable to perturbations of messages. Communication attacks in MARL has recently attracted increasing attention [3,45,47] with different focuses, as summarized in Section 5. In this paper, we consider a practical and strong threat model where malicious attackers can arbitrarily perturb a subset of communication messages during test time.…”

Section: Problem Formulation: Communication Attacks In Deployment Of ...mentioning

confidence: 99%

“…To achieve robust communication, Mitchell et al [25] adopt a Gaussian Process-based probabilistic model to compute the posterior probabilities that whether each partner is truthful. Tu et al [45] investigate the vulnerability of multi-agent autonomous systems against communication attacks, with a focus on vision tasks. Xue et al [47] propose an algorithm to defend against one adversarial communication message by an anomaly detector and a message reconstructor, which are trained with groundtruth labels and messages.…”

Section: Communication In Marl Communication Is Crucial In Solving Co...mentioning

confidence: 99%

See 1 more Smart Citation

Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems

Sun¹,

Ruijie²,

Hassanzadeh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions. However, when deploying trained communicative agents in a real-world application where noise and potential attackers exist, the safety of communication-based policies becomes a severe issue that is underexplored. Specifically, if communication messages are manipulated by malicious attackers, agents relying on untrustworthy communication may take unsafe actions that lead to catastrophic consequences. Therefore, it is crucial to ensure that agents will not be misled by corrupted communication, while still benefiting from benign communication. In this work, we consider an environment with N agents, where the attacker may arbitrarily change the communication from any C < N −1 2 agents to a victim agent. For this strong threat model, we propose a certifiable defense by constructing a message-ensemble policy that aggregates multiple randomly ablated message sets. Theoretical analysis shows that this message-ensemble policy can utilize benign communication while being certifiably robust to adversarial communication, regardless of the attacking algorithm. Experiments in multiple environments verify that our defense significantly improves the robustness of trained policies against various types of attacks.

show abstract

Adversarial Attacks On Multi-Agent Communication

Cited by 25 publications

References 27 publications

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception

Collaborative Perception for Autonomous Driving: Current Status and Future Trend

Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems

Contact Info

Product

Resources

About