Towards Robust Decision-Making for Autonomous Driving on Highway

Yang, Kai; Qiu, Sen; Jin, Shufeng; Wei, Zichun; Wang, Hong

doi:10.1109/tvt.2023.3268500

Cited by 24 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Kendall Gillies et al [45] explored combining formal methods with safe reinforcement learning to design autonomous driving systems. Kai Yang et al [9] proposed a new decision-making framework that ensures the safety lower bound by integrating traditional rule-based methods with modern machine learning techniques. Cao et al [46] introduced a new reinforcement learning algorithm that considers model confidence during the learning and decision-making processes, opting for lower-risk decisions to ensure the safety of autonomous driving decisions.…”

Section: Research On Safe Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Zhao,

Chen,

Fan

et al. 2024

Sensors

View full text Add to dashboard Cite

Reinforcement Learning (RL) methods are regarded as effective for designing autonomous driving policies. However, even when RL policies are trained to convergence, ensuring their robust safety remains a challenge, particularly in long-tail data. Therefore, decision-making based on RL must adequately consider potential variations in data distribution. This paper presents a framework for highway autonomous driving decisions that prioritizes both safety and robustness. Utilizing the proposed Replay Buffer Constrained Policy Optimization (RECPO) method, this framework updates RL strategies to maximize rewards while ensuring that the policies always remain within safety constraints. We incorporate importance sampling techniques to collect and store data in a Replay buffer during agent operation, allowing the reutilization of data from old policies for training new policy models, thus mitigating potential catastrophic forgetting. Additionally, we transform the highway autonomous driving decision problem into a Constrained Markov Decision Process (CMDP) and apply our proposed RECPO for training, optimizing highway driving policies. Finally, we deploy our method in the CARLA simulation environment and compare its performance in typical highway scenarios against traditional CPO, current advanced strategies based on Deep Deterministic Policy Gradient (DDPG), and IDM + MOBIL (Intelligent Driver Model and the model for minimizing overall braking induced by lane changes). The results show that our framework significantly enhances model convergence speed, safety, and decision-making stability, achieving a zero-collision rate in highway autonomous driving.

show abstract

Section: Research On Safe Reinforcement Learningmentioning

confidence: 99%

“…Thus, RL has become a key technology for enabling advanced automation in AVs. Its ability to adjust and optimize decision-making processes in real-time positions it at the forefront of autonomous driving policy development for AVs [9].…”

Section: Introductionmentioning

confidence: 99%

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Zhao,

Chen,

Fan

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Using the known sensors (Lidar, Radar, Camera RGB etc. ), the vehicle makes observations of the environment, the fact which makes the vehicle able to capture the current state which then enables an active decision to be made accordingly [43][44][45]. In [46], authors propose a learning method to deal with obstacle avoidance problems for autonomous vehicles in a dynamically changing environment (multiple goals).…”

Section: Learning-basedmentioning

confidence: 99%

“…Adaptive neural network [45] Good generalization performance Effective for solving the problem of nonlinear mapping…”

Section: Exponential Computational Cost Short Term Predictionmentioning

confidence: 99%

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

Abdallaoui,

Ikaouassen,

Kribèche

et al. 2023

The Journal of Engineering

View full text Add to dashboard Cite

This abstract discusses the significant progress made in autonomous vehicles, focusing on decision‐making systems and control algorithms. It explores recent advances, challenges, and contributions in the field, emphasizing the need for precise navigation and control. The paper covers various methodologies, including rule‐based methods, machine learning, deep learning, probabilistic approaches, and hybrid approaches, examining their applications and effectiveness in ensuring safe navigation. Additionally, it reviews ongoing research efforts, emerging trends, and persistent challenges related to decision‐making and manoeuvre execution in autonomous vehicles, addressing complex topics such as sensor measurement uncertainty, dynamic environment modelling, real‐time responsiveness, and safe interactions with other road users. The objective is to provide a comprehensive overview of the state of the art in autonomous vehicle navigation and control for readers.

show abstract

“…Deep Neural Networks (DNNs) have made remarkable strides in a variety of fields, ranging from autonomous driving [21], aviation [15], and healthcare [15]. While these networks have achieved extensive success, they still have drawbacks, particularly concerning their quality and reliability.…”

Section: Introductionmentioning

confidence: 99%

DeepAbstraction++: Enhancing Test Prioritization Performance via Combined Parameterized Boxes

Al-Qadasi,

Falcone,

Bensalem

2023

Bridging the Gap Between AI and Reality

View full text Add to dashboard Cite

In artificial intelligence testing, there is an increased focus on enhancing the efficiency of test prioritization methods within deep learning systems. Subsequently, the DeepAbstraction algorithm has recently become one of the leading techniques in this area. It employs a box-abstraction concept, the efficiency of which depends on the tau parameter, the clustering parameter, that influences the size of these boxes. The conclusion of the previous experiments using tau values of 0.4 or 0.05 has failed to produce optimal results among all experiments. This highlights a significant challenge in the DeepAbstraction framework concerning the appropriate selection of the tau parameter. The selection of the tau value is extremely crucial, given its decisive effect on box size and, subsequently, the stability and efficacy of the framework. Addressing this challenge, we propose a methodology called combined parameterized boxes. This approach leverages the collective verdicts of monitors with various tau values to evaluate network predictions. We assign appropriate weights to these verdicts to ensure that no single verdict influences the decision-making process, thereby ensuring balance. Furthermore, we propose multiple strategies for integrating the weighted verdicts of monitors into a conclusive verdict, such as mean, max, product, and mode. The results of our investigation demonstrate that our approach can notably boost the DeepAbstraction framework's performance. Compared to the leading algorithms, DeepAbstraction++ consistently outperforms its competitors, delivering an increase in performance between 2.38% and 7.71%. Additionally, Deep-Abstraction++ brings remarkable stability to the process, addressing a significant shortcoming of the earlier version of DeepAbstraction.

show abstract

Towards Robust Decision-Making for Autonomous Driving on Highway

Cited by 24 publications

References 30 publications

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Towards Robust Decision-Making for Autonomous Highway Driving Based on Safe Reinforcement Learning

Advancing autonomous vehicle control systems: An in‐depth overview of decision‐making and manoeuvre execution state of the art

DeepAbstraction++: Enhancing Test Prioritization Performance via Combined Parameterized Boxes

Contact Info

Product

Resources

About