End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

Abstract

Recent years have seen the rapid development of autonomous driving systems, which are typically designed in a hierarchical architecture or an end-to-end architecture. The hierarchical architecture is always complicated and hard to design, while the end-to-end architecture is more promising due to its simple structure. This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network, making it possible for the vehicle to learn end-to-end driving by itself. This paper firstly proposes an architecture for the end-to-end lane-keeping task. Unlike the traditional image-only state space, the presented state space is composed of both camera images and vehicle motion information. Then corresponding dueling neural network structure is introduced, which reduces the variance and improves sampling efficiency. Thirdly, the proposed method is applied to The Open Racing Car Simulator (TORCS) to demonstrate its great performance, where it surpasses human drivers. Finally, the saliency map of the neural network is visualized, which indicates the trained network drives by observing the lane lines. A video for the presented work is available online, https://youtu.be/76ciJmIHMD8 or https://v.youku.com/v_show/id_XNDM4ODc0MTM4NA==.html.

Study on Detection and Recognition of Traffic Lights Based on Improved YOLOv4

“…Since the detection and recognition of traffic lights in actual autonomous driving is essentially the detection of small targets in complex backgrounds, deeper research on the detection of small targets will be conducted to improve the effectiveness of algorithms in target detection under the conditions of actual autonomous driving. In addition, a novel network can be designed by inputting images and vehicle motion information simultaneously to provide a new method for the target detection of actual autonomous driving [ 37 ].…”

A Review of End-to-End Autonomous Driving in Urban Environments

“…One of the major problems in applying RL in the field of AD consists of the high-dimensional sensor inputs, as is the case of RGB images. For this reason, most of the applications of RL in AD have focused on simple driving tasks, such as lane following [7], [26]. However, in the last two years, some methods have been proposed that address this problem [14], [64]- [66].…”

“…For example, different scenarios may require different connections between modules [25], which compromises the modularity paradigm. The modular architecture is also prone to error propagation [26], in which a minor error in one module can produce catastrophic results in another, for example, a misclassification of a traffic-light can influence the decision-making process to generate a path planning that leads to a collision. Additionally, as the modules are task-specialized, they may fail to generalize to unusual conditions and unexpected situations.…”

Prediction of Soil Heavy Metal Content Based on Deep Reinforcement Learning

“…e regulatory factor should be appropriately increased to enhance the reward or punishment of environment to agent performing actions. At the later stage of training, parameters of Q network basically remain stable, and the maximum value of regulatory factor should be basically maintained to improve the model convergence rate [14,15]. us in the training process of Q network, the proportion of state value and environmental feedback reward value in total reward value should change dynamically.…”