Safe Reinforcement Learning With Stability Guarantee for Motion Planning of Autonomous Vehicles

Zhang, Lixian; Zhang, Ruixian; Wu, Tsu‐Yang; Weng, Rui; Han, Minghao; Zhao, Ye

doi:10.1109/tnnls.2021.3084685

Cited by 72 publications

(22 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The off-line gathered data with LiDAR and IMU sensors are required to update the deep neural networks. Similar collision prediction method is leveraged for safe navigation in [23]. However, collecting supervised data is arduous and it is challenging to generalize the prediction model in completely different scenarios because the off-line data are generally gathered from specific environments.…”

Section: Related Workmentioning

confidence: 99%

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Zhu

Hayashibe

2023

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

We present a hierarchical deep reinforcement learning (DRL) framework with prominent sampling efficiency and sim-to-real transfer ability for fast and safe navigation: the low-level DRL policy enables the robot to move towards the target position and keep a safe distance to obstacles simultaneously; the high-level DRL policy is supplemented to further enhance the navigation safety. We select a waypoint located on the path from the robot to the ultimate goal as the sub-goal to reduce the state space and avoid sparse reward. Moreover, the path is generated based on either a local or a global map, which can significantly improve the sampling efficiency, safety, and generalization ability of the proposed DRL framework. Additionally, a target-directed representation for the action space can be derived based on the sub-goal to improve the motion efficiency and reduce the action space. In order to demonstrate the eminent sampling efficiency, motion performance, obstacle avoidance, and generalization ability of the proposed framework, we implement sufficient comparisons with the non-learning navigation methods and DRL-based baselines, with videos, data, code, and other supplemental material shown on our website 1 .

show abstract

Section: Related Workmentioning

confidence: 99%

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Zhu

Hayashibe

2023

IEEE Trans. Ind. Electron.

View full text Add to dashboard Cite

show abstract

“…In the robotics domain, multiple approaches exist for increasing the reliability of such systems [51], [66], [75]; however, these methods are mostly heuristical in nature [1], [20], [43]. To date, existing techniques for improving the safety of robotic systems rely mostly on Lagrangian multipliers [39], [53], [57], and do not provide formal safety guarantees, but rather optimize the training in an attempt to learn the required policies [12].…”

Section: Related Workmentioning

confidence: 99%

Verifying Learning-Based Robotic Navigation Systems

Amir¹,

Corsi²,

Yerushalmi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has become a dominant deep-learning paradigm for various tasks in which complex policies are learned within reactive systems. In parallel, there has recently been significant research on verifying deep neural networks. However, to date, there has been little work demonstrating the use of modern verification tools on real, DRLcontrolled systems. In this case-study paper, we attempt to begin bridging this gap, and focus on the important task of mapless robotic navigation -a classic robotics problem, in which a robot, usually controlled by a DRL agent, needs to efficiently and safely navigate through an unknown arena towards a desired target. We demonstrate how modern verification engines can be used for effective model selection, i.e., the process of selecting the best available policy for the robot in question from a pool of candidate policies. Specifically, we use verification to detect and rule out policies that may demonstrate suboptimal behavior, such as collisions and infinite loops. We also apply verification to identify models with overly conservative behavior, thus allowing users to choose superior policies that are better at finding an optimal, shorter path to a target. To validate our work, we conducted extensive experiments on an actual robot, and confirmed that the suboptimal policies detected by our method were indeed flawed. We also compared our verification-driven approach to state-of-the-art gradient attacks, and our results demonstrate that gradient-based methods are inadequate in this setting. Our work is the first to demonstrate the use of DNN verification backends for recognizing suboptimal DRL policies in real-world robots, and for filtering out unwanted policies. We believe that the methods presented in this work can be applied to a large range of application domains that incorporate deep-learning-based agents.

show abstract

“…Only limited research has been conducted on risk-aware AV controllers. A recent study (L. Zhang et al, 2021) used the soft actor-critic (SAC) algorithm and control theorybased Lyapunov functions to train an agent subject to safety constraints. The authors in (Wen, Duan, Li, Xu, & Peng, 2020) proposed Parallel Constrained Policy Optimization (PCPO), which consists of three neural networks to estimate the policy function, value function, and a risk function.…”

Section: Risk-aware Autonomous Systemsmentioning

confidence: 99%

Safe and Efficient Manoeuvring for Emergency Vehicles in Autonomous Traffic using Multi-Agent Proximal Policy Optimisation

Parada¹,

Candela²,

Marqués³

et al. 2022

Preprint

View full text Add to dashboard Cite

Manoeuvring in the presence of emergency vehicles is still a major issue for vehicle autonomy systems. Most studies that address this topic are based on rule-based methods, which cannot cover all possible scenarios that can take place in autonomous traffic. Multi-Agent Proximal Policy Optimisation (MAPPO) has recently emerged as a powerful method for autonomous systems because it allows for training in thousands of different situations. In this study, we present an approach based on MAPPO to guarantee the safe and efficient manoeuvring of autonomous vehicles in the presence of an emergency vehicle. We introduce a risk metric that summarises the potential risk of collision in a single index. The proposed method generates cooperative policies allowing the emergency vehicle to go at 15% higher average speed while maintaining high safety distances. Moreover, we explore the trade-off between safety and traffic efficiency and assess the performance in a competitive scenario.

show abstract

Safe Reinforcement Learning With Stability Guarantee for Motion Planning of Autonomous Vehicles

Cited by 72 publications

References 28 publications

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

A Hierarchical Deep Reinforcement Learning Framework With High Efficiency and Generalization for Fast and Safe Navigation

Verifying Learning-Based Robotic Navigation Systems

Safe and Efficient Manoeuvring for Emergency Vehicles in Autonomous Traffic using Multi-Agent Proximal Policy Optimisation

Contact Info

Product

Resources

About