Visual-LiDAR Based 3D Object Detection and Tracking for Embedded Systems

Abstract

In recent years, persistent news updates on autonomous vehicles and the claims of companies entering the space, brace the notion that vehicular autonomy of level 5 is just around the corner. However, the main hindrance in asserting the full autonomy still boils down to environmental perception that affects the autonomous decisions. An efficient perceptual system requires redundancy in sensor modalities capable of performing in varying environmental conditions, and providing a reliable information using limited computational resources. This work addresses the task of 3D object detection and tracking in the vehicles' environment, using camera and 3D LiDAR as primary sensors. The proposed framework is designed to operate in an embedded system that visually classifies the objects using a lightweight neural network, while tracking is performed in 3D space using LiDAR information. The main contribution of this work is 3D LiDAR point cloud classification using visual object detector, and an IMM-UKF-JPDAF based object tracker that jointly performs 3D object detection and tracking. The performance evaluation is carried out using MOT16 metrics and ground truths provided by KITTI Datasets. Furthermore, the proposed tracker is evaluated and compared with state-of-the-art approaches. The experiments suggest that the proposed framework offers a suitable solution for embedded systems to solve 3D object detection and tracking problem, with added benefits. INDEX TERMS Kalman filter, object detection, object tracking, point cloud classification, sensor fusion.

Semantics Aware Dynamic SLAM Based on 3D MODT

“…The clustering of LiDAR point cloud is referred to as the process of grouping the closely existing LiDAR measurements, the approach adopted in this work falls under the hierarchy-based technique [ 36 ]. The 2D cylindrical representation of the non-ground LiDAR measurements from the ground segmentation module is converted to 3D cylindrical representation by vertical distribution of measurements.…”

“…To estimate the actual shape of the clustered object, box-fitting techniques are devised. In this work, considering the computational limitations and real-time requirements, a feature-based method is deployed [ 36 ]. The technique represents the cluster in a minimum rectangular shape in 2D top view, then performs L-shape point cloud fitting as shown in Figure 3 .…”

Automatic Multiple LiDAR Calibration Based on the Plane Features of Structured Environments

“…The center point of each plane = ( , , ) was converted to using the following Eq. (7). Adjacent planes are easily obtained by arranging the converted center point of the plane in descending order from the left side based on the u-axis of the image frame.…”

“…3D LiDAR has been mainly used in localization [4], simultaneous localization and mapping (SLAM) [5], object detection, and tracking [6][7]. Velodyne's VLP- 16 LiDAR has an accuracy of around 3 cm [8].…”

P2V-RCNN: Point to Voxel Feature Learning for 3D Object Detection From Point Clouds

“…T HREE Dimension (3D) point clouds captured with LiDAR sensors have been widely used for various applications, such as autonomous driving [1], [2], robotics [3] and augmented reality [4]. Different from 2D object detection that only locates the object on the 2D image, 3D object detection outputs the 3D position coordinates, 3D size, and orientation of the object in the form of a 3D bounding box.…”