Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving

Baheri, Ali; Nageshrao, Subramanya; Tseng, H. Eric; Kolmanovsky, Ilya; Girard, Anouck; Filev, Dimitar

doi:10.1109/iv47402.2020.9304744

Cited by 35 publications

(27 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, an offline method for a suitable choice of ∆ is going to be discussed. For every sampling time step i, consider wi ∈ R 8 as the difference between the car's predicted states, calculated by the linear model (6), and the car's actual states, calculated by the nonlinear vehicle model (1). For the implementation, it is important to recall that the states of the linear model ( 6) are defined as the deviation between the vehicle's states and their reference values.…”

Section: Determining the Disturbance Setmentioning

confidence: 99%

See 1 more Smart Citation

A Safe Control Architecture Based on a Model Predictive Control Supervisor for Autonomous Driving

Nezami,

Maennel,

Abbas

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents a novel, safe control architecture (SCA) for controlling an important class of systems: safety-critical systems. Ensuring the safety of control decisions has always been a challenge in automatic control. The proposed SCA aims to address this challenge by using a Model Predictive Controller (MPC) that acts as a supervisor for the operating controller, in the sense that the MPC constantly checks the safety of the control inputs generated by the operating controller and intervenes if the control input is predicted to lead to a hazardous situation in the foreseeable future invariably. Then an appropriate backup scheme can be activated, e.g., a degraded control mechanism, the transfer of the system to a safe state, or a warning signal issued to a human supervisor. For a proof of concept, the proposed SCA is applied to an autonomous driving scenario, where it is illustrated and compared in different obstacle avoidance scenarios. A major challenge of the SCA lies in the mismatch between the MPC prediction model and the real system, for which possible remedies are explored.

show abstract

Section: Determining the Disturbance Setmentioning

confidence: 99%

“…Safe decision making is particularly important in the context of path planning and control in autonomous driving vehicles (ADV), and still, an open problem [6]. The safe decision making method in [7] is based on a predictive controller for the prevention of unintended road departure.…”

Section: Introductionmentioning

confidence: 99%

A Safe Control Architecture Based on a Model Predictive Control Supervisor for Autonomous Driving

Nezami,

Maennel,

Abbas

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…-Google Scholar 14 -Engineering Village (including Compendex) 15 -Web of Science 16 -Science Direct 17 -Scopus 18 -ACM Digital Library 19 -IEEE Xplore 20 Certification of machine learning systems is a relatively new topic. To review the state-of-the-art techniques, we limited the publication date from January 2015 to September 2020 (the month when we started this work) in our searches.…”

Section: Paper Searchmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

“…Driven by both, algorithmic advances and the emergence of deep learning [1,2,3], RL has emerged from a conceptual approach to successfully tackling tasks previously deemed infeasible. This includes aspects of robotic manipulation [4,5,6,7], autonomous driving [8,9,10], and mastering combinatorially-hard board games [11,12,13,14].…”

Section: Introductionmentioning

confidence: 99%

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

Maja¹,

Wolf²,

Periyasamy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Reinforcement Learning (RL) has considerably advanced over the past decade. At the same time, state-of-the-art RL algorithms require a large computational budget in terms of training time to converge. Recent work has started to approach this problem through the lens of quantum computing, which promises theoretical speed-ups for several traditionally hard tasks. In this work, we examine a class of hybrid quantumclassical RL algorithms that we collectively refer to as variational quantum deep Q-networks (VQ-DQN). We show that VQ-DQN approaches are subject to instabilities that cause the learned policy to diverge, study the extent to which this afflicts reproduciblity of established results based on classical simulation, and perform systematic experiments to identify potential explanations for the observed instabilities. Additionally, and in contrast to most existing work on quantum reinforcement learning, we execute RL algorithms on an actual quantum processing unit (an IBM Quantum Device) and investigate differences in behaviour between simulated and physical quantum systems that suffer from implementation deficiencies. Our experiments show that, contrary to opposite claims in the literature, it cannot be conclusively decided if known quantum approaches, even if simulated without physical imperfections, can provide an advantage as compared to classical approaches. Finally, we provide a robust, universal and well-tested implementation of VQ-DQN as a reproducible testbed for future experiments.

show abstract

Deep Reinforcement Learning with Enhanced Safety for Autonomous Highway Driving

Cited by 35 publications

References 8 publications

A Safe Control Architecture Based on a Model Predictive Control Supervisor for Autonomous Driving

A Safe Control Architecture Based on a Model Predictive Control Supervisor for Autonomous Driving

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

Contact Info

Product

Resources

About