NeuroTrajectory: A Neuroevolutionary Approach to Local State Trajectory Learning for Autonomous Vehicles

Abstract

Autonomous vehicles are controlled today either based on sequences of decoupled perception-planning-action operations, either based on End2End or Deep Reinforcement Learning (DRL) systems. Current deep learning solutions for autonomous driving are subject to several limitations (e.g. they estimate driving actions through a direct mapping of sensors to actuators, or require complex reward shaping methods). Although the cost function used for training can aggregate multiple weighted objectives, the gradient descent step is computed by the backpropagation algorithm using a singleobjective loss. To address these issues, we introduce NeuroTrajectory, which is a multi-objective neuroevolutionary approach to local state trajectory learning for autonomous driving, where the desired state trajectory of the ego-vehicle is estimated over a finite prediction horizon by a perception-planning deep neural network. In comparison to DRL methods, which predict optimal actions for the upcoming sampling time, we estimate a sequence of optimal states that can be used for motion control. We propose an approach which uses genetic algorithms for training a population of deep neural networks, where each network individual is evaluated based on a multi-objective fitness vector, with the purpose of establishing a so-called Pareto front of optimal deep neural networks. The performance of an individual is given by a fitness vector composed of three elements. Each element describes the vehicle's travel path, lateral velocity and longitudinal speed, respectively. The same network structure can be trained on synthetic, as well as on real-world data sequences. We have benchmarked our system against a baseline Dynamic Window Approach (DWA), as well as against an End2End supervised learning method.

A survey of deep learning techniques for autonomous driving

“…However, Pendleton et al () do not include a review on deep learning technologies, although the state‐of‐the‐art literature has revealed an increased interest in using deep learning technologies for path planning and behavior arbitration. Following, we discuss two of the most representative deep learning paradigms for path planning, namely IL (Grigorescu, Trasnea, Marina, Vasilcoi, & Cocias, ; Rehder, Quehl, & Stiller, ; Sun, Peng, Zhan, & Tomizuka, ) and DRL‐based planning (Paxton, Raman, Hager, & Kobilarov, ; L. Yu, Shao, Wei, & Zhou, ).…”

“…The goal in IL (Grigorescu et al, ; Rehder et al, ; Sun et al, ) is to learn the behavior of a human driver from recorded driving experiences (Schwarting, Alonso‐Mora, & Rus, ). The strategy implies a vehicle teaching process from human demonstration.…”

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

“…Their approach, called Neural RRT*, is a framework for generating the sampling distribution of the optimal path under several constraints. In our previous work on local state trajectory estimation [ 4 ], we used a multi-objective neuro-evolutionary approach to train a regression-based hybrid CNN-LSTM architecture using sequences of 2D occupancy grids.…”

“…As opposed to Reference [ 4 ], we now sense the world in 3D using an octree representation, and we no longer use convolutional layers for processing the input sequences, as this intermediate representation has been taken over by the fixed state vector between the encoder and the decoder of our architecture.…”

LVD-NMPC: A learning-based vision dynamics approach to nonlinear model predictive control for autonomous vehicles

“…Although the model could learn local state sequences directly, as in the previous NeuroTrajectory work of Grigorescu et al, 24 we have chosen to learn the vision dynamics model ðc <t> ; w <t> Þ, which can be used both for state prediction in the form of ego-vehicle poses and for tuning the NMPC's quadratic cost function.…”