Urban Driving with Conditional Imitation Learning

Abstract

Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation -a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.

MP3: A Unified Model to Map, Perceive, Predict and Plan

“…Most mapless approaches [5,11,19,37,39], focus on imitating the controls of an expert driver (e.g., steering and acceleration), without providing intermediate interpretable representations that can help explain the self-driving vehicle decisions. Interpretability is of key importance in a safetycritical system particularly if a bad event was to happen.…”

DROID: Driver-centric Risk Object Identification

“…Haan et al [12] propose to incorporate functional causal models [13] into imitation learning to address the issue of "causal misidentification". In [40], they overcome the causal misidentification issue by adding noises to inputs. Our work is complementary to [12], [40].…”

“…In [40], they overcome the causal misidentification issue by adding noises to inputs. Our work is complementary to [12], [40]. Specifically, the focus of [12], [40] is to improve the robustness of driving models, whereas the proposed framework leverages driving models to determine the response of drivers in a counterfactual situation for risk object identification.…”

“…Our work is complementary to [12], [40]. Specifically, the focus of [12], [40] is to improve the robustness of driving models, whereas the proposed framework leverages driving models to determine the response of drivers in a counterfactual situation for risk object identification. We believe the two lines of work should be studied jointly and will leave for future work.…”

A Survey of End-to-End Driving: Architectures and Training Methods

“…Balancing can be achieved via upsampling the rarely occurring angles, downsampling the common ones or by weighting the samples [55]. Hawke et al [56] divide the steering angle space into bins and assign the sample weights to be equal to the bin width divided by the number of points in the bin. This leads to the few samples in sparse but wide bins having increased influence compared to samples from a densely populated narrow bin.…”