Learning Accurate and Human-Like Driving using Semantic Maps and Attention

Abstract

This paper investigates how end-to-end driving models can be improved to drive more accurately and humanlike. To tackle the first issue we exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. The maps are used in an attention mechanism that promotes segmentation confidence masks, thus focusing the network on semantic classes in the image that are important for the current driving situation. Human-like driving is achieved using adversarial learning, by not only minimizing the imitation loss with respect to the human driver but by further defining a discriminator, that forces the driving model to produce action sequences that are human-like. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data. Extensive experiments show that our driving models are more accurate and behave more human-like than previous methods.

Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization

“…It is demonstrated that our proposed method achieves a much shorter network inference time comparing to [5], as shown in Section IV-E. Moreover, to further improve the performance, similar to recent works [12], [13], we introduce a spatial attention module in our network design. The heatmap estimated by the attention module provides us a tool for visualizing and reasoning critical objects in the environment and reveals a causal relationship between them and the ego vehicle, as shown in Figure 10.…”

“…1) Imitation Learning: Imitation Learning for motion planning was first introduced in the pioneering work [14] where it directly learns a policy that maps sensor data to steering angle and acceleration. In recent years, there is an extensive literature [15], [16], [17], [18], [19], [3], [12] that follows this end-to-end philosophy. Alternatively, our work adopts a mid-to-mid approach [5], [6] which allows us to augment data handily and go beyond pure imitation by having task-specific losses.…”

Yaw-Guided Imitation Learning for Autonomous Driving in Urban Environments

“…[11] creates a multi-camera system that can offer data for a 360-degree view of the vehicle's surroundings. Furthermore, [12] uses semantic segmentation technology to improve the perception of the environment. [13] uses semantic segmentation technology while adding geometry and motion with computer vision.…”

End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

“…Even though nowadays, most autonomous driving (AD) stacks [30,48] use individual modules for perception, planning and control, end-to-end approaches have been proposed since the 80's [35] and the success of deep learning brought them back into the research spotlight [5,50]. Numerous works have studied different network architectures for this task [3,16,52], yet most of these approaches use supervised learning with expert demonstrations, which is known to suffer from covariate shift [36,40]. While data augmentation based on view synthesis [2,5,35] can partially alleviate this issue, in this paper, we tackle the problem from the perspective of expert demonstrations.…”