MLCVNet: Multi-Level Context VoteNet for 3D Object Detection

Abstract

In this paper, we address the 3D object detection task by capturing multi-level contextual information with the selfattention mechanism and multi-scale feature fusion. Most existing 3D object detection methods recognize objects individually, without giving any consideration on contextual information between these objects. Comparatively, we propose Multi-Level Context VoteNet (MLCVNet) to recognize 3D objects correlatively, building on the state-of-the-art VoteNet. We introduce three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels. Specifically, a Patch-to-Patch Context (PPC) module is employed to capture contextual information between the point patches, before voting for their corresponding object centroid points. Subsequently, an Object-to-Object Context (OOC) module is incorporated before the proposal and classification stage, to capture the contextual information between object candidates. Finally, a Global Scene Context (GSC) module is designed to learn the global scene context. We demonstrate these by capturing contextual information at patch, object and scene levels. Our method is an effective way to promote detection accuracy, achieving new state-of-the-art detection performance on challenging 3D object detection datasets, i.e., SUN RGBD and ScanNet. We also release our code at https://github.com/NUAAXQ/MLCVNet.

A benchmark dataset for canopy crown detection and delineation in co-registered airborne RGB, LiDAR and hyperspectral imagery from the National Ecological Observation Network

“…By rasterizing the annotated bounding boxes, the dataset can be used to compare segmentation strategies such as raster-based versus regional proposal networks [ 41 ] and matches more directly with polygon-based approaches to annotating crowns. Furthermore, combining 2D optical data and 3D point cloud annotations remains an active area of model development [ 42 ]. Trees have complex 3D and 2D representations and the data provided in this benchmark could be used to develop new evaluation procedures across dimensions.…”

VENet: Voting Enhancement Network for 3D Object Detection

“…3D scenes in mAP@0.25 mAP@0.5 VoteNet [28] 57.7 32.9 H3DNet [48] 60.1 39.0 LGR-Net [18] 62.2 -HGNet [3] 61.6 -SPOT [8] 60.4 36. 3 Feng [9] 59.2 -MLCVNet [41] 59.2 -VENet(Ours) 62.5 39.2 1 shows the results on ScanNet dataset using different 3D object detection methods. As shown, the proposed VENet outperforms its baseline VoteNet by 9.0% and achieves the new state-of-theart performance in the mAP@0.25 evaluation.…”

FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection

“…BRNet [4] refines voting results with the second voting focusing the representative points from the vote centers, which improves capturing the fine local structural features. MLCVNet [29] introduces three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels. H3DNet [33] improves the point group generation procedure by predicting a hybrid set of geometric primitives.…”