Exploring the Limitations of Behavior Cloning for Autonomous Driving

Abstract

Figure 1. Driving scenarios from our new benchmark where the agent needs to react to dynamic changes in the environment, handle clutter (only part of the environment is causally relevant), and predict complex sensorimotor controls (lateral and longitudinal). We show that Behavior Cloning yields state-of-the-art policies in these complex scenarios and investigate its limitations. AbstractDriving requires reacting to a wide variety of complex environment conditions and agent behaviors. Explicitly modeling each possible scenario is unrealistic. In contrast, imitation learning can, in theory, leverage data from large fleets of human-driven cars. Behavior cloning in particular has been successfully used to learn simple visuomotor policies end-to-end, but scaling to the full spectrum of driving behaviors remains an unsolved problem. In this paper, we propose a new benchmark to experimentally investigate the scalability and limitations of behavior cloning. We show that behavior cloning leads to state-of-the-art results, including in unseen environments, executing complex lateral and longitudinal maneuvers without these reactions being explicitly programmed. However, we confirm well-known limitations (due to dataset bias and overfitting), new generalization issues (due to dynamic objects and the lack of a causal model), and training instability requiring further research before behavior cloning can graduate to real-world driving. We will release our benchmark and code.

A Survey of Deep Learning Applications to Autonomous Vehicle Control

“…The authors noted that data augmentation and noise injection during training was key to learning a robust control policy. This method was further extended in [140], by using an extra module for velocity prediction, which helps the network in some situations, such as when the vehicle is stopped at a traffic light, to predict the expected vehicle velocity from visual cues and prevent it from getting stuck when the vehicle comes to a full stop. Further improvements to the model were a deeper network architecture and a larger training set, which reduced the variance in training.…”

“…This is due to the relatively small datasets used, which would cause deeper networks to overfit to the training data. However, as noted in [140], when large amounts of data are available, deeper architectures can reduce both bias and variance in training, resulting in more robust control policies. Further thought should be given to architectures specifically designed for autonomous driving, such as the conditional imitation learning model [138], where the network included a different final network layer for each high-level command used for driving.…”

Composing Diverse Policies for Temporally Extended Tasks

“…Alternatively, they can be used to calculate the relative value of each state through inverse reinforcement learning and to create a hierarchical formulation for control [19]. As explained in [20], there are limitations to BC in terms of number of demonstrations, generalization, and the challenge of modeling complex scenarios. However, we use these full task demonstrations as a means for estimating the distance to a desired goal state, which is arguably a simpler task than learning an entire policy.…”

Modeling Car-Following Behaviors and Driving Styles with Generative Adversarial Imitation Learning

“…Studies have shown that the data-driven car-following models have better generalization ability than the theoretical-driven models [ 7 , 11 ]. In the related research of data-driven models, directly applying learning methods to learn the map from states to actions is referred to as behavior cloning (BC) [ 12 ]. Although the BC models have been proven to be effective in some studies, they may suffer from the problem of cascading errors, which is very common in sequential decision-making problems [ 13 ].…”