End-to-End Driving Via Conditional Imitation Learning

Codevilla, Felipe; Miiller, Matthias; López, Antonio; Koltun, Vladlen; Dosovitskiy, Alexey

doi:10.1109/icra.2018.8460487

Cited by 885 publications

(881 citation statements)

References 26 publications

Supporting

Mentioning

815

Contrasting

Unclassified

Order By: Relevance

“…Qualitative testing was done to evaluate learning on instances from the data set, suggesting the model was learning to imitate the human driver, but no live testing was completed to validate performance. With a similar aim, Codevilla et al [138] trained a supervised learning algorithm, which uses both images and a high-level navigational command for its driving policy. The network was trained through end-to-end supervised learning, conditioned by a high-level command which could be follow road, go straight, turn left, or turn right.…”

Section: Simultaneous Lateral and Longitudinal Control Systemsmentioning

confidence: 99%

A Survey of Deep Learning Applications to Autonomous Vehicle Control

Kuutti

Bowden

Jin

et al. 2021

IEEE Trans. Intell. Transport. Syst.

451

182

View full text Add to dashboard Cite

Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalising previously learned rules to new scenarios. For these reasons, the use of deep learning for vehicle control is becoming increasingly popular. Although important advancements have been achieved in this field, these works have not been fully summarised. This paper surveys a wide range of research works reported in the literature which aim to control a vehicle through deep learning methods. Although there exists overlap between control and perception, the focus of this paper is on vehicle control, rather than the wider perception problem which includes tasks such as semantic segmentation and object detection. The paper identifies the strengths and limitations of available deep learning methods through comparative analysis and discusses the research challenges in terms of computation, architecture selection, goal specification, generalisation, verification and validation, as well as safety. Overall, this survey brings timely and topical information to a rapidly evolving field relevant to intelligent transportation systems.

show abstract

Section: Simultaneous Lateral and Longitudinal Control Systemsmentioning

confidence: 99%

A Survey of Deep Learning Applications to Autonomous Vehicle Control

Kuutti

Bowden

Jin

et al. 2021

IEEE Trans. Intell. Transport. Syst.

451

182

View full text Add to dashboard Cite

show abstract

“…To take them into account, they can be simply concatenated in the last layers of the network [16], but this requires parameterization to give them an appropriate weight. In Conditional Imitation Learning (CIL) [8], the last layers of the network are split into branches which are masked with the current navigation command, thus allowing the network to learn specific behaviors for each goal. This method is used as is in this work because it is more scalable and interpretable.…”

Section: Related Workmentioning

confidence: 99%

“…The CARLA benchmark (first introduced in [8]) is used to quantify the network performance in terms of ego vehicle trajectory prediction, and compare it to the state of the art. The benchmark regroups navigation tasks of increasing difficulty: going in a straight line, taking one turn, and navigation task with several turns without or with other road users.…”

Section: Metricsmentioning

confidence: 99%

“…To provide quantitative results on the neighbors trajectories prediction, we provide metrics on the trajectories from our test set. Mean Average Error (MAE) between predicted and ground truth trajectory points is used to quantify the Table 2: Infraction distance comparison (in km between infractions, more is better), with baselines [8,30]. Distances with > indicate that no infraction was encountered during the whole benchmark.…”

Section: Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Conditional Vehicle Trajectories Prediction in CARLA Urban Environment

Buhet

Wirbel

Perrotton

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Imitation learning is becoming more and more successful for autonomous driving. End-to-end (raw signal to command) performs well on relatively simple tasks (lane keeping and navigation). Mid-to-mid (environment abstraction to mid-level trajectory representation) or direct perception (raw signal to performance) approaches strive to handle more complex, real life environment and tasks (e.g. complex intersection). In this work, we show that complex urban situations can be handled with raw signal input and mid-level representation. We build a hybrid end-to-mid approach predicting trajectories for neighbor vehicles and for the ego vehicle with a conditional navigation goal. We propose an original architecture inspired from social pooling LSTM taking low and mid level data as input and producing trajectories as polynomials of time. We introduce a label augmentation mechanism to get the level of generalization that is required to control a vehicle. The performance is evaluated on CARLA 0.8 benchmark, showing significant improvements over previously published state of the art.

show abstract

“…In fact, those models are trained on synthetic environments in which vehicles movements are based on hard coded rules. A solution to this problem is proposed in (Shalev-Shwartz et al, 2016), where vehicles inside the simulator were trained through Imitation Learning (Codevilla et al, 2017); however, this approach is expensive since it requires a huge amount of training data. In our proposed work, this limitation has been overcome training the model in an environment populated by vehicles whose behavior has been learned in a multi-agent fashion as in (Bacchiani et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Intelligent Roundabout Insertion using Deep Reinforcement Learning

Capasso

Bacchiani

Molinari³

2020

Proceedings of the 12th International Conference on Agents and Artificial Intelligence

View full text Add to dashboard Cite

An important topic in the autonomous driving research is the development of maneuver planning systems. Vehicles have to interact and negotiate with each other so that optimal choices, in terms of time and safety, are taken. For this purpose, we present a maneuver planning module able to negotiate the entering in busy roundabouts. The proposed module is based on a neural network trained to predict when and how entering the roundabout throughout the whole duration of the maneuver. Our model is trained with a novel implementation of A3C, which we will call Delayed A3C (D-A3C), in a synthetic environment where vehicles move in a realistic manner with interaction capabilities. In addition, the system is trained such that agents feature a unique tunable behavior, emulating real world scenarios where drivers have their own driving styles. Similarly, the maneuver can be performed using different aggressiveness levels, which is particularly useful to manage busy scenarios where conservative rule-based policies would result in undefined waits.

show abstract

End-to-End Driving Via Conditional Imitation Learning

Abstract: a) Aerial view of test environment (b) Vision-based driving, view from onboard camera (c) Side view of vehicle

Cited by 885 publications

References 26 publications

A Survey of Deep Learning Applications to Autonomous Vehicle Control

A Survey of Deep Learning Applications to Autonomous Vehicle Control

Conditional Vehicle Trajectories Prediction in CARLA Urban Environment

Intelligent Roundabout Insertion using Deep Reinforcement Learning

Contact Info

Product

Resources

About