Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds

Abstract

Lidar based 3D object detection is inevitable for autonomous driving, because it directly links to environmental understanding and therefore builds the base for prediction and motion planning. The capacity of inferencing highly sparse 3D data in real-time is an ill-posed problem for lots of other application areas besides automated vehicles, e.g. augmented reality, personal robotics or industrial automation. We introduce Complex-YOLO, a state of the art real-time 3D object detection network on point clouds only. In this work, we describe a network that expands YOLOv2, a fast 2D standard object detector for RGB images, by a specific complex regression strategy to estimate multi-class 3D boxes in Cartesian space. Thus, we propose a specific Euler-Region-Proposal Network (E-RPN) to estimate the pose of the object by adding an imaginary and a real fraction to the regression network. This ends up in a closed complex space and avoids singularities, which occur by single angle estimations. The E-RPN supports to generalize well during training. Our experiments on the KITTI benchmark suite show that we outperform current leading methods for 3D object detection specifically in terms of efficiency. We achieve state of the art results for cars, pedestrians and cyclists by being more than five times faster than the fastest competitor. Further, our model is capable of estimating all eight KITTIclasses, including Vans, Trucks or sitting pedestrians simultaneously with high accuracy. Direct point cloud processing using Multi-Layer-Perceptrons [5] [10] [11] [23][24]

PointPillars: Fast Encoders for Object Detection From Point Clouds

“…In the most common paradigm the point cloud is organized in voxels and the set of voxels in each vertical column is encoded into a fixed-length, handcrafted, feature encoding to form a pseudo-image which can be processed by a standard image detection architecture. Some notable works here include MV3D [2], AVOD [11], PIXOR [30] and Complex YOLO [26] which all use variations on the same fixed encoding paradigm as the first step of their architectures. The first two methods additionally fuse the lidar features with image features to create a multimodal detector.…”

“…Following the tremendous advances in deep learning methods for computer vision, a large body of literature has investigated to what extent this technology could be applied towards object detection from lidar point clouds [31,29,30,11,2,21,15,28,26,25]. While there are many similarities between the modalities, there are two key differences: 1) the point cloud is a sparse representation, while an image is dense and 2) the point cloud is 3D, while the image is 2D.…”

“…However, the bird's eye view tends to be extremely sparse which makes direct application of convolutional neural networks impractical and inefficient. A common workaround to this problem is to partition the ground plane into a regular grid, for example 10 x 10 cm, and then perform a hand crafted feature encoding method on the points in each grid cell [2,11,26,30]. However, such methods may be sub-optimal since the hard-coded feature extraction method may not generalize to new configurations without significant engineering efforts.…”

Hierarchy Denoising Recursive Autoencoders for 3D Scene Layout Prediction

“…If the observed room contains objects that are not in the initial template, they cannot be detected. Another line of work on street and urban RGB-D data uses bird's eye view representation to capture context for 3D object detection [2,49,58]. In contrast, we operate with fused 3D point cloud data of entire rooms, and learn a generative model of 3D scene layouts from a hierarchical representation.…”

Range Adaptation for 3D Object Detection in LiDAR

“…In the most common paradigm the point cloud is organized in voxels and the set of voxels in each vertical column is encoded into a fixed-length, hand-crafted feature to form a pseudo-image which can be processed by a standard image detection architecture. Some notable works here include MV3D [2], AVOD [10], PIXOR [22], and Complex-YOLO [18], which all use variations on the same fixed encoding paradigm as the first step of their architectures. The first two methods additionally fuse the lidar features with image features to create a multimodal detector.…”