Generalizable Multi-Camera 3D Pedestrian Detection

Lima, João Paulo; Roberto, Rafael; Figueiredo, Lucas Coelho; Magalhães, Francisco Nêuton de Oliveira; Teichrieb, Verônica

doi:10.52591/lxai202106252

Cited by 5 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As it can be observed, when using the authors' AOI for evaluation, the proposed method outperforms TOP-DeepMCD by a 8.33% and DenseNet-DeepMCD by a 3.17% ranking the third best method on the Wildtrack dataset without requiring a dataset specific fine-tuning stage as the two above it. In addition, performance with respect to GMC-3D [22], which replicates the previous version of the proposed method with the addition of person re-identification features, is increased a 17.85%.…”

Section: Discussionsupporting

confidence: 53%

“…Similarly, parallel planes can be also used to create a full 3D reconstruction of pedestrians that can then be back-projected to each of the camera views, improving monocular pedestrian detection [2]. Finally, Lima et al [22] replicates a preliminar version of the method proposed in this paper, which is available as a preprint [27], with the addition of people re-identification features to guide the fusion of per-camera detections.…”

Section: Fusion and Refinement Of Per-camera Detectionsmentioning

confidence: 99%

See 1 more Smart Citation

Semantic-driven multi-camera pedestrian detection

et al. 2022

View full text Add to dashboard Cite

In the current worldwide situation, pedestrian detection has reemerged as a pivotal tool for intelligent video-based systems aiming to solve tasks such as pedestrian tracking, social distancing monitoring or pedestrian mass counting. Pedestrian detection methods, even the top performing ones, are highly sensitive to occlusions among pedestrians, which dramatically degrades their performance in crowded scenarios. The generalization of multi-camera setups permits to better confront occlusions by combining information from different viewpoints. In this paper, we present a multi-camera approach to globally combine pedestrian detections leveraging automatically extracted scene context. Contrarily to the majority of the methods of the state-of-the-art, the proposed approach is scene-agnostic, not requiring a tailored adaptation to the target scenario–e.g., via fine-tuning. This noteworthy attribute does not require ad hoc training with labeled data, expediting the deployment of the proposed method in real-world situations. Context information, obtained via semantic segmentation, is used (1) to automatically generate a common area of interest for the scene and all the cameras, avoiding the usual need of manually defining it, and (2) to obtain detections for each camera by solving a global optimization problem that maximizes coherence of detections both in each 2D image and in the 3D scene. This process yields tightly fitted bounding boxes that circumvent occlusions or miss detections. The experimental results on five publicly available datasets show that the proposed approach outperforms state-of-the-art multi-camera pedestrian detectors, even some specifically trained on the target scenario, signifying the versatility and robustness of the proposed method without requiring ad hoc annotations nor human-guided configuration.

show abstract

Section: Discussionsupporting

confidence: 53%

Section: Fusion and Refinement Of Per-camera Detectionsmentioning

confidence: 99%

Semantic-driven multi-camera pedestrian detection

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In this part, we discuss the selection of key parameters in the proposed ReID algorithm first, and then compare the proposed framework with a state-of-the-art method [ 26 ]. We all utilize AlphaPose as detector and clustering-based algorithms during inter-camera ReID.…”

Section: Methodsmentioning

confidence: 99%

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

Hachiuma

Saitō

et al. 2022

J. Imaging

View full text Add to dashboard Cite

Multi-camera multi-person (MCMP) tracking and re-identification (ReID) are essential tasks in safety, pedestrian analysis, and so on; however, most research focuses on outdoor scenarios because they are much more complicated to deal with occlusions and misidentification in a crowded room with obstacles. Moreover, it is challenging to complete the two tasks in one framework. We present a trajectory-based method, integrating tracking and ReID tasks. First, the poses of all surgical members captured by each camera are detected frame-by-frame; then, the detected poses are exploited to track the trajectories of all members for each camera; finally, these trajectories of different cameras are clustered to re-identify the members in the operating room across all cameras. Compared to other MCMP tracking and ReID methods, the proposed one mainly exploits trajectories, taking texture features that are less distinguishable in the operating room scenario as auxiliary cues. We also integrate temporal information during ReID, which is more reliable than the state-of-the-art framework where ReID is conducted frame-by-frame. In addition, our framework requires no training before deployment in new scenarios. We also created an annotated MCMP dataset with actual operating room videos. Our experiments prove the effectiveness of the proposed trajectory-based ReID algorithm. The proposed framework achieves 85.44% accuracy in the ReID task, outperforming the state-of-the-art framework in our operating room dataset.

show abstract

“…We expect the availability of such large-scale multi-camera multiple people tracking dataset will encourage more participants in this research topic. This dataset is also valuable for the evaluation of other tasks, such as multi-view people detection [20,28] and monocular multiple people tracking [7]. To summarize, our contributions are as follows:…”

Section: Rgb Tracking Labelsmentioning

confidence: 99%

“…3D pose can be estimated [10,36] by merging 2D skeleton estimations from multiple 2D camera views, using a 3D regression network or graph matching. Meanwhile, multi-view person detection approaches [19,20,28,34] also utilize camera calibration to merge multiple 2D detections or features to generate more reliable 3D person detection results. The accuracy of these approaches heavily depends on the quality of the 2D person detection or 2D pose estimation.…”

Section: Related Workmentioning

confidence: 99%

MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark

Han¹,

You²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-camera tracking systems are gaining popularity in applications that demand high-quality tracking results, such as frictionless checkout because monocular multi-object tracking (MOT) systems often fail in cluttered and crowded environments due to occlusion. Multiple highly overlapped cameras can significantly alleviate the problem by recovering partial 3D information. However, the cost of creating a high-quality multi-camera tracking dataset with diverse camera settings and backgrounds has limited the dataset scale in this domain. In this paper, we provide a largescale densely-labeled multi-camera tracking dataset in five different environments with the help of an auto-annotation system. The system uses overlapped and calibrated depth and RGB cameras to build a high-performance 3D tracker that automatically generates the 3D tracking results. The 3D tracking results are projected to each RGB camera view using camera parameters to create 2D tracking results. Then, we manually check and correct the 3D tracking results to ensure the label quality, which is much cheaper than fully manual annotation. We have conducted extensive experiments using two real-time multi-camera trackers and a person re-identification (ReID) model with different settings. This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments. Also, our results demonstrate that adapting the trackers and ReID models on this dataset significantly improves their performance. Our dataset will be publicly released upon the acceptance of this work.

show abstract

Generalizable Multi-Camera 3D Pedestrian Detection

Cited by 5 publications

References 17 publications

Semantic-driven multi-camera pedestrian detection

Semantic-driven multi-camera pedestrian detection

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room

MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark

Contact Info

Product

Resources

About