Immortal Tracker: Tracklet Never Dies

Wang, Qitai; Chen, Yuntao; Pang, Ziqi; Wang, Naiyan; Zhang, Zhaoxiang

doi:10.48550/arxiv.2111.13672

Cited by 4 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the recent advances in LiDAR-based 3D detection [25,68], especially the reliable range information, most state-of-theart 3D MOT algorithms adopt a "tracking-by-detection" paradigm [63]. Given single frame detection outputs, different approaches have been proposed to improve data association [42,65,70], motion propagation [9,75], and life cycling [42,58]. However, most of these works assume the location accuracy of each detection output.…”

Section: Related Workmentioning

confidence: 99%

Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

Pang¹,

Li²,

Tokmakov³

et al. 2023

Preprint

View full text Add to dashboard Cite

This work proposes an end-to-end multi-camera 3D multi-object tracking (MOT) framework. It emphasizes spatio-temporal continuity and integrates both past and future reasoning for tracked objects. Thus, we name it "Pastand-Future reasoning for Tracking" (PF-Track). Specifically, our method adapts the "tracking by attention" framework and represents tracked instances coherently over time with object queries. To explicitly use historical cues, our "Past Reasoning" module learns to refine the tracks and enhance the object features by cross-attending to queries from previous frames and other objects. The "Future Reasoning" module digests historical information and predicts robust future trajectories. In the case of long-term occlusions, our method maintains the object positions and enables re-association by integrating motion predictions. On the nuScenes dataset, our method improves AMOTA by a large margin and remarkably reduces ID-Switches by 90% compared to prior approaches, which is an order of magnitude less. The code and models are made available at https://github.com/TRI-ML/PF-Track.

show abstract

Section: Related Workmentioning

confidence: 99%

Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

Pang¹,

Li²,

Tokmakov³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The combination of GIoU and NMS preprocessing improves the tracking result. ImmortalTracker [12] and PC3T [13] consider the similar idea, silently maintaining the tracks even when the tracks are no longer visible, and reducing the ID switches and fragmented tracks in tracking results. The aforementioned trackers terminate or initiate tracks based on hard-coded rules that might be too rigid for the MOT application.…”

Section: Related Workmentioning

confidence: 99%

GNN-PMB: A Simple but Effective Online 3D Multi-Object Tracker without Bells and Whistles

Liu¹,

Bai²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

Multi-object tracking (MOT) is among crucial applications in modern advanced driver assistance systems (ADAS) and autonomous driving (AD) systems. Most solutions to MOT are based on random vector Bayesian filters like global nearest neighbor (GNN) plus rule-based heuristical track maintenance. With the development of random finite set (RFS) theory, the RFS Bayesian filters have been applied in MOT tasks for ADAS and AD systems recently. However, their usefulness in the real traffic is open to doubt due to computational cost and implementation complexity. In this paper, it is revealed that GNN with rulebased heuristic track maintenance is insufficient for LiDARbased MOT tasks in ADAS and AD systems. This judgement is illustrated by systematically comparing several different multipoint object filter-based tracking frameworks, including traditional random vector Bayesian filters with rule-based heuristical track maintenance and RFS Bayesian filters. Moreover, a simple and effective tracker, namely Poisson multi-Bernoulli filter using global nearest neighbor (GNN-PMB) tracker, is proposed for LiDAR-based MOT tasks. The proposed GNN-PMB tracker achieves competitive results in nuScenes test dataset, and shows superior tracking performance over other state-of-the-art LiDAR only trackers and LiDAR and camera fusion-based trackers.

show abstract

“…Hallucinating the occluded positions is necessary to operate forecasting models on all the agents. Our baseline uses a Kalman filter to predict occluded positions, inspired by how 3D multi-object tracking addresses occlusions [26] [29]. Overview of predictive streamers.…”

Section: Algorithms a Pipeline Of Streaming Forecastingmentioning

confidence: 99%

“…Occlusion reasoning is better with multi-modality. We compare our estimation of occluded positions with previous 3D perception strategies that capitalize occlusion reasoning [26][25] [29]. The main distinction is that we utilize multi-modal predictions.…”

Section: Quantitative Analysismentioning

confidence: 99%

“…Our predictive streamer estimates the states of occluded agents to enable robust predictions for them. While inferring occluded positions is a long-standing problem in 3D perception [26][25] [29], existing strategies of using the Kalman filter or single-modal trajectories are insufficient for the forecasting purpose, because the forecaster depends on the estimated occluded positions and amplifies the errors in occlusion reasoning. We discover that a more advanced strategy of utilizing multi-modal trajectories predicted by the forecaster performs better, as it provides higher-quality estimation covering a larger spectrum of motion distributions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning hierarchical behavior and motion planning for autonomous driving

Wang

Zhang

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Trajectory forecasting is a widely-studied problem for autonomous navigation. However, existing benchmarks evaluate forecasting based on independent snapshots of trajectories, which are not representative of real-world applications that operate on a continuous stream of data. To bridge this gap, we introduce a benchmark that continuously queries future trajectories on streaming data and we refer to it as "streaming forecasting." Our benchmark inherently captures the disappearance and re-appearance of agents, presenting the emergent challenge of forecasting for occluded agents, which is a safetycritical problem yet overlooked by snapshot-based benchmarks. Moreover, forecasting in the context of continuous timestamps naturally asks for temporal coherence between predictions from adjacent timestamps. Based on this benchmark, we further provide solutions and analysis for streaming forecasting. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster. Our algorithm estimates the states of occluded agents by propagating their positions with multimodal trajectories, and leverages differentiable filters to ensure temporal consistency. Both occlusion reasoning and temporal coherence strategies significantly improve forecasting quality, resulting in 25% smaller endpoint errors for occluded agents and 10-20% smaller fluctuations of trajectories. Our work is intended to generate interest within the community by highlighting the importance of addressing motion forecasting in its intrinsic streaming setting. Code is available at https://github.com/ziqipang/StreamingForecasting.

show abstract

Immortal Tracker: Tracklet Never Dies

Cited by 4 publications

References 22 publications

Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking

GNN-PMB: A Simple but Effective Online 3D Multi-Object Tracker without Bells and Whistles

Learning hierarchical behavior and motion planning for autonomous driving

Contact Info

Product

Resources

About