A Bayesian Filter for Multi-View 3D Multi-Object Tracking With Occlusion Handling

Abstract

This paper proposes an online multi-camera multi-object tracker that only requires monocular detector training, independent of the multi-camera configurations, allowing seamless extension/deletion of cameras without retraining effort. The proposed algorithm has a linear complexity in the total number of detections across the cameras, and hence scales gracefully with the number of cameras. It operates in the 3D world frame, and provides 3D trajectory estimates of the objects. The key innovation is a high fidelity yet tractable 3D occlusion model, amenable to optimal Bayesian multi-view multi-object filtering, which seamlessly integrates, into a single Bayesian recursion, the sub-tasks of track management, state estimation, clutter rejection, and occlusion/misdetection handling. The proposed algorithm is evaluated on the latest WILDTRACKS dataset, and demonstrated to work in very crowded scenes on a new dataset.

Generalizable Multi-Camera 3D Pedestrian Detection

“…The pipeline to estimate the 3D location of pedestrians in multi-camera scenarios, in a generalizable manner, often employs 2D monocular pedestrian detectors [7,15,14] and later fuse their results based on multi-view properties [16,23,17]. In this scenario, one way to make 3D pedestrian detection robust to domain shift, therefore generalizable, is to use monocular person detectors that do not need retraining for a specific target domain [19,9,12,7].…”

“…In this scenario, one way to make 3D pedestrian detection robust to domain shift, therefore generalizable, is to use monocular person detectors that do not need retraining for a specific target domain [19,9,12,7]. Another advantage of monocular detectors re-use is to simplify setup requirements, easing cameras addition/removal/combination [17].…”

Generalizable Multi-Camera 3D Pedestrian Detection

“…The pipeline to estimate the 3D location of pedestrians in multi-camera scenarios, in a generalizable manner, often employs 2D monocular pedestrian detectors [7,15,14] and later fuse their results based on multi-view properties [16,23,17]. In this scenario, one way to make 3D pedestrian detection robust to domain shift, therefore generalizable, is to use monocular person detectors that do not need retraining for a specific target domain [19,9,12,7].…”

“…In this scenario, one way to make 3D pedestrian detection robust to domain shift, therefore generalizable, is to use monocular person detectors that do not need retraining for a specific target domain [19,9,12,7]. Another advantage of monocular detectors re-use is to simplify setup requirements, easing cameras addition/removal/combination [17].…”

Occlusion Handling in Generic Object Detection: A Review

“…With partial occlusion, the deep neural network based classifiers are less robust compared to humans [6], and it worsens the performance of detectors [7]. Therefore, occlusion handling has been studied extensively such as in pedestrian detection [8] [9] [10] [11], object tracking [12] [13] [14], face detection [15] [16], stereo images [17], car detection [18] [19], semantic part detection [20] [21], etc. However due to a huge number of variations in object category and instances, occlusion handling in generic object detection from a single still image is much harder.…”