Deep Reinforcement Learning With NMPC Assistance Nash Switching for Urban Autonomous Driving

Alighanbari, Sina; Azad, Nasser L.

doi:10.1109/tiv.2022.3167616

Cited by 11 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [109], a nonlinear MPC (NMPC) controller was proposed to speed up the training of the DDPG algorithm. A major concern for training DRL algorithms is low sample efficiency, which means that such algorithms have to be trained for a large number of episodes, significantly increasing training time.…”

Section: Roundaboutsmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

2023

Self Cite

View full text Add to dashboard Cite

The development of autonomous vehicles (AVs) is becoming increasingly important as the need for reliable and safe transportation grows. However, in order to achieve level 5 autonomy, it is crucial that such AVs can navigate through complex and unconventional scenarios. It has been observed that currently deployed AVs, like human drivers, struggle the most in cases of adverse weather conditions, unsignalized intersections, crosswalks, roundabouts, and near-accident scenarios. This review paper provides a comprehensive overview of the various navigation methodologies used in handling these situations. The paper discusses both traditional planning methods such as graph-based approaches and emerging solutions including machine-learning based approaches and other advanced decision-making and control techniques. The benefits and drawbacks of previous studies in this area are discussed in detail and it is identified that the biggest shortcomings and challenges are benchmarking, ensuring interpretability, incorporating safety as well as road user interactions, and unrealistic simplifications such as the availability of accurate and perfect perception information. Some suggestions to tackle these challenges are also presented.

show abstract

Section: Roundaboutsmentioning

confidence: 99%

“…In [110], similar to [109], the authors proposed the use of an NMPC controller during the training phase of the DDPG agent. However, in that case, the NMPC controller was used for the safe exploration of a DDPG agent during both the training as well as test phases.…”

Section: Roundaboutsmentioning

confidence: 99%

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…In recent years, computer vision technology has made significant strides, impacting fields like autonomous driving and target monitoring. , It has also been increasingly applied to hydrological monitoring, including tasks such as identifying water bodies and assessing water quality using remote sensing images, measuring river flow velocity using spatiotemporal images, and detecting water levels through video images . Computer vision technology has demonstrated potential in automatic water turbidity monitoring as well.…”

Section: Introductionmentioning

confidence: 99%

An Approach Based on Video Image Intelligent Recognition for Water Turbidity Monitoring in River

Zhou,

Chen,

Chen

et al. 2024

ACS EST Water

View full text Add to dashboard Cite

Water turbidity stands as a pivotal water quality indicator for water quality monitoring, but conventional contact monitoring equipment is expensive and challenging to maintain. The development of a cost-effective, noncontact turbidity measurement method remains paramount for scientific water quality monitoring. In this study, we propose a video image-based model for water turbidity recognition incorporating image regression algorithms. Residual neural networks (ResNet) and transfer learning are employed to enhance model robustness, and the balanced mean squared error (BMSE) loss function is utilized to address the impact of imbalanced training data. An image preclassification module is implemented to mitigate the influence of changing lighting conditions on turbidity recognition. The effectiveness of these methods is validated using data from the United States Geological Survey (USGS). Our findings reveal that the BMSE loss function outperforms the mean squared error (MSE) loss function in turbidity recognition. Global image recognition consistently surpasses region of interest (ROI) image recognition, providing more accurate and reliable results under various lighting scenarios. Additionally, the influence of neural network depth on recognition accuracy varies with lighting conditions, highlighting the need for a careful selection of network architectures. Despite the promising results, the study identifies challenges in continuous noncontact monitoring, especially during sunrise and sunset, where rapidly changing lighting conditions can lead to recognition errors. The video image-based water turbidity identification model established in the study provides a new way to carry out low-cost and noncontact monitoring of water quality in rivers.

show abstract

“…Autonomous overtaking is one of the most common yet challenging driving maneuvers, improving trip efficiency by avoiding a slower or stationary preceding vehicle (see [1][2][3][4][5][6][7][8][9][10]). In the early years of autonomous vehicle research, the primary focus was predominantly on trajectory planning and tracking.…”

Section: Introductionmentioning

confidence: 99%

High‐level decision‐making for autonomous overtaking: An MPC‐based switching control approach

Wang,

Chen,

Jiang

et al. 2024

IET Intelligent Trans Sys

View full text Add to dashboard Cite

The key motivation of this paper lies in the development of a high‐level decision‐making framework for autonomous overtaking maneuvers on two‐lane country roads with dynamic oncoming traffic. To generate an optimal and safe decision sequence for such scenario, an innovative high‐level decision‐making framework that combines model predictive control (MPC) and switching control methodologies is introduced. Specifically, the autonomous vehicle is abstracted and modelled as a switched system. This abstraction allows vehicle to operate in different modes corresponding to different high‐level decisions. It establishes a crucial connection between high‐level decision‐making and low‐level behaviour of the autonomous vehicle. Furthermore, barrier functions and predictive models that account for the relationship between the autonomous vehicle and oncoming traffic are incorporated. This technique enables us to guarantee the satisfaction of constraints, while also assessing performance within a prediction horizon. By repeatedly solving the online constrained optimization problems, we not only generate an optimal decision sequence for overtaking safely and efficiently but also enhance the adaptability and robustness. This adaptability allows the system to respond effectively to potential changes and unexpected events. Finally, the performance of the proposed MPC framework is demonstrated via simulations of four driving scenarios, which shows that it can handle multiple behaviours.

show abstract

Deep Reinforcement Learning With NMPC Assistance Nash Switching for Urban Autonomous Driving

Cited by 11 publications

References 30 publications

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review

An Approach Based on Video Image Intelligent Recognition for Water Turbidity Monitoring in River

High‐level decision‐making for autonomous overtaking: An MPC‐based switching control approach

Contact Info

Product

Resources

About