Learning how to drive in a real world simulation with deep Q-Networks

Wolf, Peter; Hubschneider, Christian; Weber, Michael; Bauer, André; Hartl, Jonathan; Dürr, F.; Zöllner, J. Marius

doi:10.1109/ivs.2017.7995727

Cited by 105 publications

(61 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The direction of motion of a car can be estimated using cameras mounted on it. Studies have also examined self-driving using the deep Q-network [19] where a Q-value is updated through simulations. A car autonomously runs using the Q-value, and learning is needed to update it.…”

Section: Research On Driving Methods Of Carsmentioning

confidence: 99%

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

et al. 2018

View full text Add to dashboard Cite

Recent research on deep learning has been applied to a diversity of fields. In particular, numerous studies have been conducted on self-driving vehicles using end-to-end approaches based on images captured by a single camera. End-to-end controls learn the output vectors of output devices directly from the input vectors of available input devices. In other words, an end-to-end approach learns not by analyzing the meaning of input vectors, but by extracting optimal output vectors based on input vectors. Generally, when end-to-end control is applied to self-driving vehicles, the steering wheel and pedals are controlled autonomously by learning from the images captured by a camera. However, high-resolution images captured from a car cannot be directly used as inputs to Convolutional Neural Networks (CNNs) owing to memory limitations; the image size needs to be efficiently reduced. Therefore, it is necessary to extract features from captured images automatically and to generate input images by merging the parts of the images that contain the extracted features. This paper proposes a learning method for end-to-end control that generates input images for CNNs by extracting road parts from input images, identifying the edges of the extracted road parts, and merging the parts of the images that contain the detected edges. In addition, a CNN model for end-to-end control is introduced. Experiments involving the Open Racing Car Simulator (TORCS), a sustainable computing environment for cars, confirmed the effectiveness of the proposed method for self-driving by comparing the accumulated difference in the angle of the steering wheel in the images generated by it with those of resized images containing the entire captured area and cropped images containing only a part of the captured area. The results showed that the proposed method reduced the accumulated difference by 0.839% and 0.850% compared to those yielded by the resized images and cropped images, respectively.

show abstract

Section: Research On Driving Methods Of Carsmentioning

confidence: 99%

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

et al. 2018

View full text Add to dashboard Cite

show abstract

“…David improved the classical multi-agent reinforcement learning algorithm and used the neural network and the kernel smoothing technology to perform the approximate greedy operation on the unknown environment, which could generate multiple alternative paths [17]. Wolf searched for various extreme conditions through reinforcement learning, learned driving behavior in simulation and trained the autopilot system to plan a safe driving path [18]. To sum up, the path planning algorithm can be categorized into global path planning and local path planning, also known as motion planning.…”

Section: Related Researchmentioning

confidence: 99%

Anti-rollover Motion Planning for Automated Heavy Trucks

Jin

Wang

et al. 2020

Preprint

View full text Add to dashboard Cite

Anti-rollover is an important performance for automated heavy trucks, which has been seldomly considered in the motion planning. This paper proposes an anti-rollover motion planning based on model predictive control (MPC) for automated heavy trucks. Taking the coupling of roll motion of sprung mass of the front axle with that of the drive axle into consideration, a seven degrees of freedom rollover dynamics model is established, and an evaluation index that can accurately describe the rollover motion is derived for heavy trucks. Then, a model predictive control strategy is designed for motion planning that combines the rollover dynamics, the artificial potential field for obstacle avoidance, and the trajectory tracking. In addition, the optimal path is calculated that considers collision avoidance, anti-rollover and vehicle dynamic constraints. Furthermore, three typical scenarios are applied to validate the performance of the proposed motion planning algorithm. The obtained results demonstrate that the proposed anti-rollover motion planning can effectively avoid collisions and reduce the rollover risk simultaneously when confronting edge scenarios.

show abstract

“…Wolf et al . [WHW*17] presented a Deep Q‐Networks to steer a vehicle in 3D physics simulations. In this approach, the goal of a vehicle is to follow the lane to complete laps on arbitrary courses, and an action‐based reward function is motivated by a potential in real word reinforcement learning scenarios.…”

Section: Applications In Autonomous Drivingmentioning

confidence: 99%

A Survey on Visual Traffic Simulation: Models, Evaluations, and Applications in Autonomous Driving

Chao

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Virtualized traffic via various simulation models and real‐world traffic data are promising approaches to reconstruct detailed traffic flows. A variety of applications can benefit from the virtual traffic, including, but not limited to, video games, virtual reality, traffic engineering and autonomous driving. In this survey, we provide a comprehensive review on the state‐of‐the‐art techniques for traffic simulation and animation. We start with a discussion on three classes of traffic simulation models applied at different levels of detail. Then, we introduce various data‐driven animation techniques, including existing data collection methods, and the validation and evaluation of simulated traffic flows. Next, we discuss how traffic simulations can benefit the training and testing of autonomous vehicles. Finally, we discuss the current states of traffic simulation and animation and suggest future research directions.

show abstract

Learning how to drive in a real world simulation with deep Q-Networks

Cited by 105 publications

References 22 publications

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Anti-rollover Motion Planning for Automated Heavy Trucks

A Survey on Visual Traffic Simulation: Models, Evaluations, and Applications in Autonomous Driving

Contact Info

Product

Resources

About