Monocular Quasi-Dense 3D Object Tracking

Hu, Hanwen; Yang, Yung-Hsu; Fischer, Tobias; Darrell, Trevor; Yu, Fisher; Sun, Min

doi:10.1109/tpami.2022.3168781

Cited by 61 publications

(52 citation statements)

References 82 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The 3D tracking task aims at following each detected object and to keep track of their trajectories in the real world space. As for 2D MOT current research largely focuses on the tracking-by-detection paradigm [6,32,52,57,60], including state-of-the-art trackers [17,18,34,54]. [52] sets a simple baseline using a combination of a 3D Kalman filter and the Hungarian algorithm [24].…”

Section: Related Workmentioning

confidence: 99%

“…[52] sets a simple baseline using a combination of a 3D Kalman filter and the Hungarian algorithm [24]. QD-3DT [17,18] proposes an unified framework for joint detection, appearance feature extraction and tracking. It combines appearance and predicted motion information to match detections with tracklets.…”

Section: Related Workmentioning

confidence: 99%

“…We use an existing 3D object detector from the current leading camera-only 3D tracking method on nuScenes to fairly compare with them. We apply our tracking pipeline on raw detections from the QD-3DT [18] object detector which is based on Faster RCNN [35]. The object detector is trained on our target datasets and it performs predictions on monocular images.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…Some state-of-the-art methods perform joint detection and tracking but at the expense of weaker objects representations or lack of synergy between motion and appearance cues. Some also manually design similarity metrics for the data association step, not fully exploiting the power of Neural Networks [6,17,18,62]. We instead advocate for a data-driven method that focuses on strong appearance and motion representations.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM

Marinello

Proesmans

Gool

2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

3D object tracking is a critical task in autonomous driving systems. It plays an essential role for the system's awareness about the surrounding environment. At the same time there is an increasing interest in algorithms for autonomous cars that solely rely on inexpensive sensors, such as cameras. In this paper we investigate the use of triplet embeddings in combination with motion representations for 3D object tracking. We start from an off-the-shelf 3D object detector, and apply a tracking mechanism where objects are matched by an affinity score computed on local object feature embeddings and motion descriptors. The feature embeddings are trained to include information about the visual appearance and monocular 3D object characteristics, while motion descriptors provide a strong representation of object trajectories. We will show that our approach effectively re-identifies objects, and also behaves reliably and accurately in case of occlusions, missed detections and can detect re-appearance across different field of views. Experimental evaluation shows that our approach outperforms state-of-the-art on nuScenes by a large margin. We also obtain competitive results on KITTI.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM

Marinello

Proesmans

Gool

2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

show abstract

“…In Table 3, the proposed method is compared with Cen-terTrack [4], the best camera-based tracker QD-3DT [26] and CFTrack [27]. A considerable improvement of 3.9% in AMOTA score against the camera-based tracker QD-3DT and 5.6% against CFTrack which is also based on CenterFusion detection results is achieved.…”

Section: Comparison Regarding Modalities Radar-cameramentioning

confidence: 99%

PTMOT: A Probabilistic Multiple Object Tracker Enhanced by Tracklet Confidence for Autonomous Driving

et al. 2022

View full text Add to dashboard Cite

Real driving scenarios, due to occlusions and disturbances, provide disordered and noisy measurements, which makes the task of multi-object tracking quite challenging. Conventional approach is to find deterministic data association; however, it has unstable performance in high clutter density. This paper proposes a novel probabilistic tracklet-enhanced multiple object tracker (PTMOT), which integrates Poisson multi-Bernoulli mixture (PMBM) filter with confidence of tracklets. The proposed method is able to realize efficient and robust probabilistic association for 3D multi-object tracking (MOT) and improve the PMBM filter’s continuity by smoothing single target hypothesis with global hypothesis. It consists of two key parts. First, the PMBM tracker based on sets of tracklets is implemented to realize probabilistic fusion of disordered measurements. Second, the confidence of tracklets is smoothed through a smoothing-while-filtering approach. Extensive MOT tests on nuScenes tracking dataset demonstrate that the proposed method achieves superior performance in different modalities.

show abstract

Tracking Every Thing in the Wild

Liu

Danelljan

Ding

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The ability to recognize, localize and track dynamic objects in a scene is fundamental to many real-world applications, such as self-driving and robotic systems. Yet, traditional multiple object tracking (MOT) benchmarks rely only on a few object categories that hardly represent the multitude of possible objects that are encountered in the real world. This leaves contemporary MOT methods limited to a small set of pre-defined object categories. In this paper, we address this limitation by tackling a novel task, openvocabulary MOT, that aims to evaluate tracking beyond predefined training categories. We further develop OVTrack, an open-vocabulary tracker that is capable of tracking arbitrary object classes. Its design is based on two key ingredients: First, leveraging vision-language models for both classification and association via knowledge distillation; second, a data hallucination strategy for robust appearance feature learning from denoising diffusion probabilistic models. The result is an extremely data-efficient open-vocabulary tracker that sets a new state-of-the-art on the large-scale, largevocabulary TAO benchmark, while being trained solely on static images.

show abstract

Monocular Quasi-Dense 3D Object Tracking

Cited by 61 publications

References 82 publications

TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM

TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM

PTMOT: A Probabilistic Multiple Object Tracker Enhanced by Tracklet Confidence for Autonomous Driving

Tracking Every Thing in the Wild

Contact Info

Product

Resources

About