Point Siamese Network for Person Tracking Using 3D Point Clouds

Cui, Yubo; Fang, Zheng; Zhou, Sifan

doi:10.3390/s20010143

Cited by 9 publications

(5 citation statements)

References 27 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, it ignores the local geometric information of each target proposal. PSN [7] leverages 3D Siamese network for single person tracking. However, it cannot predict the orientation and size of the target.…”

Section: Related Workmentioning

confidence: 99%

A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds

Yan¹,

Wu²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent works on 3D single object tracking treat the tracking as a target-specific 3D detection task, where an off-the-shelf 3D detector is commonly employed for tracking. However, it is non-trivial to perform accurate targetspecific detection since the point cloud of objects in raw LiDAR scans is usually sparse and incomplete. In this paper, we address this issue by explicitly leveraging temporal motion cues and propose DMT, a Detector-free Motion prediction based 3D Tracking network that totally removes the usage of complicated 3D detectors, which is lighter, faster, and more accurate than previous trackers. Specifically, the motion prediction module is firstly introduced to estimate a potential target center of the current frame in a pointcloud free way. Then, an explicit voting module is proposed to directly regress the 3D box from the estimated target center. Extensive experiments on KITTI and NuScenes datasets demonstrate that our DMT, without applying any complicated 3D detectors, can still achieve better performance (∼10% improvement on the NuScenes dataset) and faster tracking speed (i.e., 72 FPS) than state-of-the-art approaches. Our codes will be released publicly.

show abstract

Section: Related Workmentioning

confidence: 99%

A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds

Yan¹,

Wu²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Zarzar et al [81] generated efficient proposals with a siamese network from the BEV representation of point clouds, after which it tracks 3D objects in accordance with the ROI-wise appearance information regularized by the latter siamese framework. PSN [82] first extracts features through a shared PointNet-like framework and then conducts feature augmentation and the attention mechanism through two separate branches to generate a similarity map so as to match the patches. Recently, MLVSNet [83] proposes conducting Hough voting on multi-level features of target and search area instead of only on final features to overcome insufficient target detection in sparse point clouds.…”

Section: Lidaronly Trackingmentioning

confidence: 99%

“…DualBranch [72] mvPC Bbox growing method + multi-hypothesis extended Kalman filter PV-RCNN [73] pPC & vPC Voxel-to-keypoint 3D scene encoding + keypoint-to-grid RoI feature abstraction P2B [74] pPC Target-specific feature augmentation + 3D target proposal and verification CenterPoint [76] pillar/vPC Map-view feature representation + center-based anchor-free head SC-ST [80] pPC Siamese tracker(resemble the latent space of a shape completion network) BEV-ST [81] mvPC Efficient RPN+Siamese tracker PSN [82] pPC Siamese tracker(feature extraction + attention module + feature augumentation) MLVSNet [83] pPC Multi-level voting+Target-Guided Attention+Vote-cluster Feature Enhancement BAT [84] pPC Box-aware feature fusion + box-aware tracker…”

Section: Lidar -Onlymentioning

confidence: 99%

Deep learning for LiDAR-only and LiDAR-fusion 3D perception: a survey

Wu¹,

Liang²,

Chen³

2022

Intell Robot

View full text Add to dashboard Cite

The perception system for robotics and autonomous cars relies on the collaboration among multiple types of sensors to understand the surrounding environment. LiDAR has shown great potential to provide accurate environmental information, and thus deep learning on LiDAR point cloud draws increasing attention. However, LiDAR is unable to handle severe weather. The sensor fusion between LiDAR and other sensors is an emerging topic due to its supplementary property compared to a single LiDAR. Challenges exist in deep learning methods that take LiDAR point cloud fusion data as input, which need to seek a balance between accuracy and algorithm complexity due to data redundancy. This work focuses on a comprehensive survey of deep learning on LiDAR-only and LiDAR-fusion 3D perception tasks. Starting with the representation of LiDAR point cloud, this paper then introduces its unique characteristics and the evaluation dataset as well as metrics. This paper gives a review according to four key tasks in the field of LiDAR-based perception: object classification, object detection, object tracking, and segmentation (including semantic segmentation and instance segmentation). Finally, we present the overlooked aspects of the current algorithms and possible solutions, hoping this paper can serve as a reference for the related research.

show abstract

“…Therefore, it is difficult to be deployed on small robot systems, like person following robots or drones. Cui et al [10] also proposed a Point Siamese Network for person tracking. However, the method only predicts the position (x,y,z coordinates) of the target but no orientation and size information are predicted.…”

mentioning

confidence: 99%

“…Recently, Qi et al [12] adopted a Siamese Network to tackle 3D object tracking based on VoteNet [13]. However, their method has similar problems to [10], which could not estimate the size information of objects that are important in real-scene applications.…”

mentioning

confidence: 99%

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

et al. 2021

Self Cite

View full text Add to dashboard Cite

3D single object tracking is a key issue for autonomous following robot, where the robot should robustly track and accurately localize the target for efficient following. In this paper, we propose a 3D tracking method called 3D-SiamRPN Network to track a single target object by using raw 3D point cloud data. The proposed network consists of two subnetworks. The first subnetwork is feature embedding subnetwork which is used for point cloud feature extraction and fusion. In this subnetwork, we first use PointNet++ to extract features of point cloud from template and search branches. Then, to fuse the information of features in the two branches and obtain their similarity, we propose two cross correlation modules, named Pointcloud-wise and Point-wise respectively. The second subnetwork is region proposal network(RPN), which is used to get the final 3D bounding box of the target object based on the fusion feature from cross correlation modules. In this subnetwork, we utilize the regression and classification branches of a region proposal subnetwork to obtain proposals and scores, thus get the final 3D bounding box of the target object. Experimental results on KITTI dataset show that our method has a competitive performance in both Success and Precision compared to the state-of-the-art methods, and could run in real-time at 20.8 FPS. Additionally, experimental results on H3D dataset demonstrate that our method also has good generalization ability and could achieve good tracking performance in a new scene without re-training. Index Terms-3D single object tracking, LIDAR point-cloud, siamese network, region proposal network. I. INTRODUCTIONO BJECT tracking, which is a key issue in computer vision and robotics, could be used in a wide range of applications, such as augmented reality, self-driving cars and mobile robotics. Traditionally, cameras are widely used for object tracking, known as visual object tracking (VOT). For mobile robots or self-driving cars, 3D object tracking is usually more Manuscript

show abstract

Point Siamese Network for Person Tracking Using 3D Point Clouds

Cited by 9 publications

References 27 publications

A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds

A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds

Deep learning for LiDAR-only and LiDAR-fusion 3D perception: a survey

3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud

Contact Info

Product

Resources

About