ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA

Abstract

This paper presents a field-programmable gate array (FPGA) design of a segmentation algorithm based on convolutional neural network (CNN) that can process light detection and ranging (LiDAR) data in real-time. For autonomous vehicles, drivable region segmentation is an essential step that sets up the static constraints for planning tasks. Traditional drivable region segmentation algorithms are mostly developed on camera data, so their performance is susceptible to the light conditions and the qualities of road markings. LiDAR sensors can obtain the 3D geometry information of the vehicle surroundings with high precision. However, it is a computational challenge to process a large amount of LiDAR data in real-time. In this paper, a convolutional neural network model is proposed and trained to perform semantic segmentation using data from the LiDAR sensor. An efficient hardware architecture is proposed and implemented on an FPGA that can process each LiDAR scan in 17.59 ms, which is much faster than the previous works. Evaluated using Ford and KITTI road detection benchmarks, the proposed solution achieves both high accuracy in performance and real-time processing in speed.

3D Vehicle Detection on an FPGA from LiDAR Point Clouds

“…One of the biggest differences when programming an application that runs on a GPU or on a FPGA is the fact that available memory to handle the multiple meta-parameters and weights during CNNs training is more constrained in a FPGA than in a GPU or CPU. Therefore, one of the breakthroughs of this work is the implementation of a simulated quantization step needed to run the training and validation on an FPGA based on a similar approach as the one presented in [10]. As explained there, when using CPU or GPU floating-point operation are used, which create gradients during training.…”

“…In the method presented in this work a convolutional block similar than the one proposed by [10] has been implemented. One of the main issues of the deep neural network implementation is the so-called vanishing gradient, which means that the gradient of the error used in the back-propagation during training to update the weights gets smaller and smaller on each layer.…”

“…However, lately precise 32 or 64 channel LIDAR sensors have come out in the market with reasonable price and size that make it more reasonable to be integrated in a commercial vehicle. As stated in works such as [10], although graphics processing units (GPUs) are more suitable for parallel processing they do need a high power consumption, which could make them a bottle-neck for the integration of deep learning algorithms into vehicles, as they have limited power supply. In this scenario, FPGA are a low-power consumption option more suitable for embedded applications as they can be programmed as a customized integrated circuit that is able to perform massive parallel processing and data communications on-chip.…”

Road Segmentation using CNN and Distributed LSTM

“…In recent years, growing research interest is witnessed in automated driving systems (ADS) and advanced driverassistance systems (ADAS). As one of the essential modules, road segmentation perceives the surroundings, detects the drivable region and builds an occupancy map [1] [2] [3] [4]. A drivable region is a connected road surface area that is not occupied by any vehicles, pedestrians, cyclists or other obstacles.…”

An Interactive LiDAR to Camera Calibration

“…Automated driving systems (ADS) and advanced driver assistant systems (ADAS) equipped on intelligent vehicles rely on multiple sensors to perceive their surroundings. In recent research works, LiDAR-based algorithms have shown their advantage on drivable region segmentation [8] [9], object detection [18], and simultaneous localization and mapping [19] [15]. LIDARs are also fused with cameras to improve the accuracy of 3D object detection [2].…”