Efficient and Robust LiDAR-Based End-to-End Navigation

Abstract

Deep learning has been used to demonstrate endto-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate information, existing end-to-end driving solutions are mainly based on cameras since processing 3D data requires a large memory footprint and computation cost. On the other hand, increasing the robustness of these systems is also critical; however, even estimating the model's uncertainty is very challenging due to the cost of sampling-based methods. In this paper, we present an efficient and robust LiDAR-based end-to-end navigation framework. We first introduce Fast-LiDARNet that is based on sparse convolution kernel optimization and hardwareaware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass and then fuses the control predictions intelligently. We evaluate our system on a full-scale vehicle and demonstrate lane-stable as well as navigation capabilities. In the presence of out-of-distribution events (e.g., sensor failures), our system significantly improves robustness and reduces the number of takeovers in the real world.

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

“…Deep neural networks (DNNs) have revolutionized the field of artificial intelligence (AI) and have delivered impressive performance in computer vision [118,155,263], natural language processing [22,71,271,291] and speech recognition [69,122,328]. They can be applied in various real-world scenarios, such as mobile phones [131,217,329], self-driving cars [5,20,59,194] and smart hospitals [116,183,338]. However, their superior performance comes at the cost of high computational complexity.…”

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

“…The complexity of the task is exacerbated when dealing with unstructured and extreme outdoor environments, as in the case of planetary exploration [1], forest inventory [2], search and rescue tasks [3] or precision agriculture [4]. Learning-based control models have been successfully applied to tackle navigation tasks [5], [6], [7] in structured environments; however, learning-based sensorimotor control in unknown, variable, and highly cluttered real-world scenes is much more complex than indoor or controlled settings, and it is still an open research area [8]. To generalize well to real-world applications, these approaches require a significant amount of training data, covering a multitude of variable conditions and scenes.…”

RREV: A Robust and Reliable End-to-End Visual Navigation

“…This method not only requires accurate intersection location of vehicles, but the preset steering command cannot be applied to other complex intersections, either. Other studies tend to use navigation maps to provide steering information [7][8][9]. The deep learning model needs to learn the steering information from the navigation map and fuse it with other sensor information to predict the control variable.…”

“…The development of the research on the evaluation of uncertainty of a neural network [10] has brought the impetus to solve this problem. The evidence depth learning method [8,11] places a prior distribution on the category probability, treats the prediction of the neural network as subjective opinions, and learns the function that collects the evidence leading to these opinions by a deterministic neural net from data. The Bayesian neural network(BNN) method [12][13][14] puts the prior value above the network weight and estimates the prediction uncertainty by approximating the moment of the posterior prediction distribution.…”

“…The potential uncertainty of the network propagation is used to reconstruct the image to detect the new image which is not included in the training distribution. Liu et al [8] proposed hybrid evidential fusion, which can learn the uncertainty of prediction directly. By training another branch of the network, the evidence distribution is output for each prediction, which is designed to capture evidence (or confidence) related to any decision made by the network.…”