Safe Reinforcement Learning for Autonomous Lane Changing Using Set-Based Prediction

Abstract

Machine learning approaches often lack safety guarantees, which are often a key requirement in real-world tasks. This paper addresses the lack of safety guarantees by extending reinforcement learning with a safety layer that restricts the action space to the subspace of safe actions. We demonstrate the proposed approach using lane changing in autonomous driving. To distinguish safe actions from unsafe ones, we compare planned motions with the set of possible occupancies of traffic participants generated by set-based predictions. In situations where no safe action exists, a verified failsafe controller is executed. We used real-world highway traffic data to train and test the proposed approach. The evaluation result shows that the proposed approach trains agents that do not cause collisions during training and deployment.

Safe Reinforcement Learning Using Black-Box Reachability Analysis

“…One can also use prior knowledge as an inductive bias for the exploration process [4], [15]; for example, one can provide a finite set of demonstrations as guidance on the task [16]. Although these approaches can provide strong safety guarantees, most of them assume prior knowledge on some or all components of the system model [17], [18], [19], which is not always feasible for more complicated systems. In addition, some techniques in this category also suffer the curse of dimensionality [18], [20].…”

Autonomous Racing with Multiple Vehicles using a Parallelized Optimization with Safety Guarantee using Control Barrier Functions

“…For competitive scenarios like autonomous lane change or lane merge, both model-based methods [1] and learningbased methods [2] have been demonstrated to generate the ego vehicle's desired trajectory. Similarly, control using model-based methods [3], [4] and learning-based methods [5] have also been developed. However, the criteria to evaluate planning and control performance are different for car racing compared to autonomous driving on public roads.…”

Risk Conditioned Neural Motion Planning

“…In addition to constrained MDP-based approaches, there exist works that improve safety by generating more samples in the risky region to bootstrap performance in critical scenarios [18]; by using a safety layer at the end of a deep neural network to verify the safety of the resulting policy and replacing with a backup safe action if needed [19]; by proposing a reachability-based trajectory safe guard to ensure the safety of a policy [20], etc. In this paper, we focus on generating risk-bounded policies directly by modeling the risk as an explicit constraint in the objective function.…”