What data do we need for training an AV motion planner?

Abstract

We investigate what grade of sensor data is required for training an imitation-learning-based AV planner on human expert demonstration. Machine-learned planners [1] are very hungry for training data, which is usually collected using vehicles equipped with the same sensors used for autonomous operation [1]. This is costly and non-scalable. If cheaper sensors could be used for collection instead, data availability would go up, which is crucial in a field where data volume requirements are large and availability is small. We present experiments using up to 1000 hours worth of expert demonstration and find that training with 10x lower-quality data outperforms 1x AV-grade data in terms of planner performance (see Fig. 1). The important implication of this is that cheaper sensors can indeed be used. This serves to improve data access and democratize the field of imitation-based motion planning. Alongside this, we perform a sensitivity analysis of planner performance as a function of perception range, field-of-view, accuracy, and data volume, and reason about why lower-quality data still provide good planning results.

Quantity over Quality: Training an AV Motion Planner with Large Scale Commodity Vision Data

“…This indicates that we can achieve highperformance autonomy systems using large quantities of data that is an order of magnitude cheaper to collect, rendering an Autonomy 2.0 [1] approach financially viable. We also note that our previous work [2], estimating of the value of commodity vision data via simulation (grey) was very close to reality (green). sensors would make the approach viable, but these lowerfidelity data sources come at a cost: increased technical complexity.…”

“…However, this approach scales poorly with the high-dimensional space of driving situations. This has motivated recent interest in imitation-learning [3], [4], [5], [2], i.e. using machine-learned (ML) models trained to mimic human behaviors from real-world driving examples.…”

“…For example, [3] used millions of short driving demonstrations, while [6] used tens of thousands of human driven miles. This poses a new challenge, since these driving demonstrations are typically collected using large fleets of vehicles equipped with expensive HD sensor suites: LiDARs, RADARs, and cameras [3], [2]. Data collection at scale with these expensive sensors is not a financially or operationally viable strategy.…”

“…• To the best of our knowledge, this is the first study showing it is possible to train a motion planner with data from sensors that are an order of magnitude lower-cost than those used at test time. Previous work analysed this problem space in simulation [2], i.e by adding noise to the accurate output of a perception system operating on HD sensors to make it look like the lower-fidelity output we would get from cheaper sensors. We compare these predictions to our real-world results (Fig.…”

Autonomy 2.0: Why is self-driving always 5 years away?

“…4.2). This is corroborated by recent results suggesting that training ML planners on large amounts of data with lower accuracy perception might be preferable to smaller amounts with higher accuracy [46]. As the sensor prices stand today, this points to the camera only (and optionally sparse LIDAR) as the most promising solution to strike a balance between cost and fidelity, but this remains an open problem.…”