Deep Reinforcement Learning for Time Optimal Velocity Control using Prior Knowledge

Hartmann, Gabriel; Shiller, Zvi; Azaria, Amos

doi:10.1109/ictai.2019.00034

Cited by 23 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Trajectory tracking can be done using feedback control by minimizing position error, [18], [31], Model Predictive Control (MPC) by generating a sequence of open loop commands that minimize tracking error subject to dynamic constraints [11], [27], and learning-based control that iteratively minimizes lap time and tracking error [32], [33].…”

Section: Trajectory Trackingmentioning

confidence: 99%

Competitive Driving of Autonomous Vehicles

2022

View full text Add to dashboard Cite

This paper addresses the issue of autonomous competitive yet safe driving in the context of the Indy Autonomous Challenge (IAC) simulation race. The IAC is the first multi-vehicle autonomous head-tohead competition, reaching speeds of 300 km/h along an oval track modeled after the Indianapolis Motor Speedway (IMS). We present a racing controller that attempts to maximize progress along the track while avoiding collisions with opponent vehicles and obeying the race rules. To this end, the racing controller first computes a race line offline. During the race, it repeatedly computes a small set of dynamically feasible maneuver candidates, each tested for collision with the opponent vehicles. It then selects a collision-free maneuver that maximizes the progress along the track and obeys the race rules. Our controller was tested in a 6-vehicle simulation, managing to run competitively with no collision over 30 laps. In addition, it managed to drive within a close range of the leading vehicle during most of the IAC final simulation race.

show abstract

Section: Trajectory Trackingmentioning

confidence: 99%

Competitive Driving of Autonomous Vehicles

2022

View full text Add to dashboard Cite

show abstract

“…Reinforcement learning based methods have recently shown great success in many domains, including Atari games [17], Go [22], and autonomous vehicles [9,10,19,21]. Human-aided reinforcement learning introduces methods that enable the reinforcement learning agent to take advantage of human knowledge in order to learn more efficiently.…”

Section: Related Workmentioning

confidence: 99%

Criticality-Based Varying Step-Number Algorithm for Reinforcement Learning

Spielberg¹,

Azaria²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

In the context of reinforcement learning we introduce the concept of criticality of a state, which indicates the extent to which the choice of action in that particular state influences the expected return. That is, a state in which the choice of action is more likely to influence the final outcome is considered as more critical than a state in which it is less likely to influence the final outcome.We formulate a criticality-based varying step number algorithm (CVS) -a flexible step number algorithm that utilizes the criticality function provided by a human, or learned directly from the environment. We test it in three different domains including the Atari Pong environment, Road-Tree environment, and Shooter environment. We demonstrate that CVS is able to outperform popular learning algorithms such as Deep Q-Learning and Monte Carlo.

show abstract

“…Reinforcement learning algorithms have been shown to be capable of tackling problems in autonomous driving systems [33][34][35]. The authors of [36] invoked a beneficial combination of deep reinforcement learning (DRL) and numerical solutions for controlling the instantaneous velocity of the AV. As a result, the maximal possible speed was obtained, while collisions were avoided.…”

Section: A State-of-the-artmentioning

confidence: 99%

Enhancing the Fuel-Economy of V2I-Assisted Autonomous Driving: A Reinforcement Learning Approach

Liu

Yue

et al. 2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

A novel framework is proposed for enhancing the driving safety and fuel economy of autonomous vehicles (AVs) with the aid of vehicle-to-infrastructure (V2I) communication networks. The problem of driving trajectory design is formulated for minimizing the total fuel consumption, while enhancing driving safety (by obeying the traffic rules and avoiding obstacles). In an effort to solve this pertinent problem, a deep reinforcement learning (DRL) approach is proposed for making collision-free decisions. Firstly, a deep Q-network (DQN) aided algorithm is proposed for determining the trajectory and velocity of the AV by receiving real-time traffic information from the base stations (BSs). More particularly, the AV acts as an agent to carry out optimal action such as lane change and velocity change by interacting with the environment. Secondly, to overcome the large overestimation of action values by the Q-learning model, a double deep Q-network (DDQN) algorithm is proposed by decomposing the max-Q-value operation into action selection and action evaluation. Additionally, three practical driving policies are also proposed as benchmarks. Numerical results are provided for demonstrating that the proposed trajectory design algorithms are capable of enhancing the driving safety and fuel economy of AVs. We demonstrate that the proposed DDQN based algorithm outperforms the DQN based algorithm. Additionally, it is also demonstrated that the proposed fuel-economy (FE) based driving policy derived from the DRL algorithm is capable of achieving in excess of 24% of fuel savings over the benchmarks.

show abstract

Deep Reinforcement Learning for Time Optimal Velocity Control using Prior Knowledge

Cited by 23 publications

References 20 publications

Competitive Driving of Autonomous Vehicles

Competitive Driving of Autonomous Vehicles

Criticality-Based Varying Step-Number Algorithm for Reinforcement Learning

Enhancing the Fuel-Economy of V2I-Assisted Autonomous Driving: A Reinforcement Learning Approach

Contact Info

Product

Resources

About