What You See is What You Get: Exploiting Visibility for 3D Object Detection

Hu, Pengxiang; Ziglar, Jason; Held, David; Ramanan, Deva

doi:10.1109/cvpr42600.2020.01101

Cited by 124 publications

(60 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During the measurement process, the detected LiDAR points are the result of a physical raycasting. When representing the LiDAR points using pillar features, one fundamentally neglects the hidden model information of observability, including information on free space and occupied areas [8]. However, we argue that the observability information might be beneficial for the dense top-view segmentation.…”

Section: A Point Cloud Feature Encodingmentioning

confidence: 88%

“…The commonly used 2D occupancy grid map encodes the occupancy probability for each evenly spaced grid cell on the ground plane. Since the mapping of 3D measurements to 2D implies a loss of information in the height, [8] further divides the 3D world into a set of 3D voxels and encodes the occupancy information for each voxel to obtain 3D occupancy grid maps. In addition to occupancy, other features such as intensity, density and observations can also be derived to form multi-layer grid maps [9].…”

Section: A Point Cloud Representationmentioning

confidence: 99%

See 1 more Smart Citation

PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

Fei

Peng

Heidenreich³

et al. 2021

2021 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Section: A Point Cloud Feature Encodingmentioning

confidence: 88%

Section: A Point Cloud Representationmentioning

confidence: 99%

PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

Fei

Peng

Heidenreich³

et al. 2021

2021 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

“…3D Object Detection Modern LiDAR-based 3D object detectors can be organized into three sub-categories based on the way they represent the input point cloud: i.e., voxelization-based detectors [55,8,21,50,19,60,47,68,58,20,64,56], point-based methods [45,63,32,38,62,46] as well as hybrid methods [67,61,5,12,44]. Besides input representation, aggregating points across frames [13,65,14,41], using additional input modalities [19,4,39,57,25,29,48,37], and multi-task training [27,59,30,24] have also been studied to boost the performance. Despite such progress in model design, the output representation and evaluation metrics have remained mostly unchanged.…”

Section: Related Workmentioning

confidence: 99%

Revisiting 3D Object Detection From an Egocentric Perspective

Deng¹,

Qi²,

Najibi³

et al. 2021

Preprint

View full text Add to dashboard Cite

3D object detection is a key module in safety-critical robotics applications such as autonomous driving. For such applications, we care the most about how the detections impact the ego-agent's behavior and safety (the egocentric perspective). Intuitively, we seek more accurate descriptions of object geometry when it's more likely to interfere with the ego-agent's motion trajectory. However, current detection metrics, based on box Intersection-over-Union (IoU), are object-centric and are not designed to capture the spatio-temporal relationship between objects and the ego-agent. To address this issue, we propose a new egocentric measure to evaluate 3D object detection: Support Distance Error (SDE). Our analysis based on SDE reveals that the egocentric detection quality is bounded by the coarse geometry of the bounding boxes. Given the insight that SDE can be improved by more accurate geometry descriptions, we propose to represent objects as amodal contours, specifically amodal star-shaped polygons, and devise a simple model, StarPoly, to predict such contours. Our experiments on the large-scale Waymo Open Dataset show that SDE better reflects the impact of detection quality on the ego-agent's safety compared to IoU; and the estimated contours from StarPoly consistently improve the egocentric detection quality over recent 3D object detectors.

show abstract

“…In our literature search, we find only Huang et al [16] have tackled this issue previously. Ngiam et al [24] and Hu et al [15] also consider multiple 3D frames as input, but both use relatively simple techniques of reusing seed points or concatenating input over multiple frames.…”

Section: Introductionmentioning

confidence: 99%

“…Sequences 0,1,3,4,5,9,11,12,15,17,19,20 were used for training, while the remaining were chosen for validation.…”

mentioning

confidence: 99%

3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation

Sharma,

Lim

2021

Preprint

View full text Add to dashboard Cite

3D object detection using LiDAR data remains a key task for applications like autonomous driving and robotics. Unlike in the case of 2D images, LiDAR data is almost always collected over a period of time. However, most work in this area has focused on performing detection independent of the temporal domain. In this paper we present 3D-FCT, a Siamese network architecture that utilizes temporal information to simultaneously perform the related tasks of 3D object detection and tracking. The network is trained to predict the movement of an object based on the correlation features of extracted keypoints across time. Calculating correlation across keypoints only allows for real-time object detection. We further extend the multi-task objective to include a tracking regression loss. Finally, we produce high accuracy detections by linking short-term object tracklets into long term tracks based on the predicted tracks. Our proposed method is evaluated on the KITTI tracking dataset where it is shown to provide an improvement of 5.57% mAP over a state-of-the-art approach.

show abstract

What You See is What You Get: Exploiting Visibility for 3D Object Detection

Cited by 124 publications

References 16 publications

PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Data

Revisiting 3D Object Detection From an Egocentric Perspective

3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation

Contact Info

Product

Resources

About