MLVSNet: Multi-level Voting Siamese Network for 3D Visual Tracking

Abstract

Benefiting from the excellent performance of Siamesebased trackers, huge progress on 2D visual tracking has been achieved. However, 3D visual tracking is still underexplored. Inspired by the idea of Hough voting in 3D object detection, in this paper, we propose a Multi-level Voting Siamese Network (MLVSNet) for 3D visual tracking from outdoor point cloud sequences. To deal with sparsity in outdoor 3D point clouds, we propose to perform Hough voting on multi-level features to get more vote centers and retain more useful information, instead of voting only on the final level feature as in previous methods. We also design an efficient and lightweight Target-Guided Attention (TGA) module to transfer the target information and highlight the target points in the search area. Moreover, we propose a Vote-cluster Feature Enhancement (VFE) module to exploit the relationships between different vote clusters. Extensive experiments on the 3D tracking benchmark of KITTI dataset demonstrate that our MLVSNet outperforms state-of-the-art methods with significant margins. Code will be available at https://github.com/CodeWZT/MLVSNet.

A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds

“…Following this, BAT [45] proposes to add the bounding box information provided in the first frame as an additional cue. MLVS-Net [33] performs Hough voting on multi-level features for getting more vote centers. PTT [30] introduces the transformer architecture to enhance the target-specific feature extracted in P2B.…”

“…Before going to the details of our proposed explicit voting module (EVM), we give a short review on the RPN module (VoteNet) used by previous trackers [27,30,33,45]. The architecture of VoteNet includes two aspects: 1) Hough voting to convert the search area seeds into possible target centers; and 2) cluster neighboring possible target centers to obtain the final target center.…”

“…The tailored dataset contains 21 outdoor scenes and 8 categories of targets. Following [27,30,33,45], we generate tracklets and split the KITTI training set into three parts: scenes 00-16, scenes 17-18, and scenes 19-20 for training/valid/test set respectively since the annotations of the test set in KITTI are inaccessible. Furthermore, we also conduct experiments on the more challenging dataset NuScenes [5].…”

“…We compare our network with the state-of-the-art methods: SC3D [1], its follow-up SC3D-RPN [41], FSiamese [47], 3DSiamRPN [8], P2B [27], MLVSNet [33], PTT [30], and BAT [45]. For a fair comparison, we use the same evaluation metrics.…”

“…Following this, BAT [45] explores the free box information to enhance the target-specific search feature. MLVSNet [33] proposes to perform voting on multi-level features to get more vote centers. With breakthroughs of transformer-based vision methods, the authors of PTT [30] introduce a transformer module to further refine the target-specific search feature.…”