Adversarial Reinforcement Learning Framework for Benchmarking Collision Avoidance Mechanisms in Autonomous Vehicles

Behzadan, Vahid; Munir, Arslan

doi:10.48550/arxiv.1806.01368

Cited by 3 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter is concerned with multi-agent RL settings, in which agents aim to maximize their returns in competition with other agents. While some DRL security problems can be modeled as adversarial RL (e.g., [18]), this cannot be generalized as the adversary is not necessarily a learning agent.…”

Section: Threat Modelmentioning

confidence: 99%

“…An adversary may perturb the environment and its configuration by some vector δ t (i.e, s t = s t + δ t ) to manipulate the training or inference of the agent. For instance, Behzadan & Munir [18] demonstrate that through sequential reconfiguration of obstacles on a road, an adversary can manipulate the trajectory of a DRL-based autonomous vehicle at test-time.…”

Section: Drl Attack Surfacementioning

confidence: 99%

“…In this work, the ultrasonic collision avoidance of the robot was manipulated via artificial ultrasonic "pings" that would allow the attacker to manipulate the trajectory of the robot. Within the domain of autonomous navigation, Behzadan & Munir [18] propose an adversarial DRL agent specifically trained to manipulate the operating environment of an autonomously navigating DRL agent and induce collisions or trajectory manipulations by exploiting the collision avoidance policy of the target. In this attack, the adversarial DRL agent is trained as another autonomous navigation agent with a reward function that incorporates adversarial objectives, such as pursuing trajectories that will lead to the target colliding with itself or other objects in the environment.…”

Section: Test-time Attacksmentioning

confidence: 99%

“…The security problem is however concerned with settings in which an adversarial element intentionally seeks to compromise the natural operation of the system for malicious purposes. In the example of autonomous navigation, an agent with a provably safe RL policy can be forced into collisions via means such as the intentional perturbation of sensory input [18], as detailed in the following sections. Hence, while there are overlapping areas between RL safety and security, a different approach is required to study the security problem.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning

Behzadan,

Munir

2018

Preprint

Self Cite

View full text Add to dashboard Cite

Since the inception of Deep Reinforcement Learning (DRL) algorithms, there has been a growing interest in both research and industrial communities in the promising potentials of this paradigm. The list of current and envisioned applications of deep RL ranges from autonomous navigation and robotics to control applications in the critical infrastructure, air traffic control, defense technologies, and cybersecurity. While the landscape of opportunities and the advantages of deep RL algorithms are justifiably vast, the security risks and issues in such algorithms remain largely unexplored. To facilitate and motivate further research on these critical challenges, this paper presents a foundational treatment of the security problem in DRL. We formulate the security requirements of DRL, and provide a high-level threat model through the classification and identification of vulnerabilities, attack vectors, and adversarial capabilities. Furthermore, we present a review of current literature on security of deep RL from both offensive and defensive perspectives. Lastly, we enumerate critical research venues and open problems in mitigation and prevention of intentional attacks against deep RL as a roadmap for further research in this area.

show abstract

Section: Threat Modelmentioning

confidence: 99%

Section: Drl Attack Surfacementioning

confidence: 99%

Section: Test-time Attacksmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning

Behzadan,

Munir

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results suggested this could be an effective alternative to manual testing of complex software controllers. In more recent work, Behzadan & Munir [25] demonstrated that a reinforcement learning agent could be trained to create collisions with other road vehicles, by training an agent to collide against two agents, a DNN and a rule-based system. The number of episodes to convergence and minimum time-to-collision were then used to argue the DNN was the safer control policy.…”

Section: Introductionmentioning

confidence: 99%

Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies

Kuutti¹,

Fallah²,

Bowden³

2020

Preprint

View full text Add to dashboard Cite

Deep learning has become an increasingly common technique for various control problems, such as robotic arm manipulation, robot navigation, and autonomous vehicles. However, the downside of using deep neural networks to learn control policies is their opaque nature and the difficulties of validating their safety. As the networks used to obtain stateof-the-art results become increasingly deep and complex, the rules they have learned and how they operate become more challenging to understand. This presents an issue, since in safety-critical applications the safety of the control policy must be ensured to a high confidence level. In this paper, we propose an automated black box testing framework based on adversarial reinforcement learning. The technique uses an adversarial agent, whose goal is to degrade the performance of the target model under test. We test the approach on an autonomous vehicle problem, by training an adversarial reinforcement learning agent, which aims to cause a deep neural networkdriven autonomous vehicle to collide. Two neural networks trained for autonomous driving are compared, and the results from the testing are used to compare the robustness of their learned control policies. We show that the proposed framework is able to find weaknesses in both control policies that were not evident during online testing and therefore, demonstrate a significant benefit over manual testing methods.

show abstract

Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning

Ilahi¹,

Usama²,

Qadir³

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) has numerous applications in the real world thanks to its outstanding ability in quickly adapting to the surrounding environments. Despite its great advantages, DRL is susceptible to adversarial attacks, which precludes its use in real-life critical systems and applications (e.g., smart grids, traffic controls, and autonomous vehicles) unless its vulnerabilities are addressed and mitigated. Thus, this paper provides a comprehensive survey that discusses emerging attacks in DRL-based systems and the potential countermeasures to defend against these attacks. We first cover some fundamental backgrounds about DRL and present emerging adversarial attacks on machine learning techniques. We then investigate more details of the vulnerabilities that the adversary can exploit to attack DRL along with the state-of-the-art countermeasures to prevent such attacks. Finally, we highlight open issues and research challenges for developing solutions to deal with attacks for DRL-based intelligent systems.

show abstract

Adversarial Reinforcement Learning Framework for Benchmarking Collision Avoidance Mechanisms in Autonomous Vehicles

Cited by 3 publications

References 0 publications

The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning

The Faults in Our Pi Stars: Security Issues and Open Challenges in Deep Reinforcement Learning

Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies

Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning

Contact Info

Product

Resources

About