JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection

Abstract

Voxel and point representations are widely applied in recent 3D object detection tasks from LiDAR point clouds. Voxel representations contribute to efficiently and rapidly locating objects, whereas point representations are capable of describing intra-object spatial relationship for detection refinement. In this work, we aim to exploit the strengths of both two representations, and present a novel two-stage detector, named Joint Point-Voxel Network (JPV-Net). Specifically, our framework is equipped with a Dual Encoders-Fusion Decoder, which consists of the dual encoders to extract voxel features of sketchy 3D scenes and point features rich in geometric context, respectively, and the Feature Propagation Fusion (FP-Fusion) decoder to attentively fuse them from coarse to fine. By making use of the advantages of these features, the refinement network can effectively eliminate false detection and provide better accuracy. Besides, to further develop the perception characteristics of voxel CNN and point backbone, we design two novel intersection-over-union (IoU) estimation modules for proposal generation and refinement, both of which can alleviate the misalignment between the localization and the classification confidence. Extensive experiments on the KITTI dataset and ONCE dataset demonstrate that our proposed JPV-Net outperforms other state-of-the-art methods with remarkable margins.

Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views

“…Explicit Representations Explicit representation methods commonly employ point clouds (Ran, Liu, and Wang 2022;Huang et al 2023), voxels (Sitzmann et al 2019;Song, Jiang, and Yao 2022), meshes (Feng et al 2019;Yang, Qiu, and Fu 2023), or MPI (Zhou et al 2018;Kundu et al 2020) to represent 3D geometry and appearance. Despite their computational efficiency, these techniques often pose optimization challenges due to their discontinuous nature.…”