Perceiving Humans: From Monocular 3D Localization to Social Distancing

Abstract

Perceiving humans in the context of Intelligent Transportation Systems (ITS) often relies on multiple cameras or expensive LiDAR sensors. In this work, we present a new cost-effective vision-based method that perceives humans' locations in 3D and their body orientation from a single image. We address the challenges related to the ill-posed monocular 3D tasks by proposing a neural network architecture that predicts confidence intervals in contrast to point estimates. Our neural network estimates human 3D body locations and their orientation with a measure of uncertainty. Our proposed solution (i) is privacy-safe, (ii) works with any fixed or moving cameras, and (iii) does not rely on ground plane estimation. We demonstrate the performance of our method with respect to three applications: locating humans in 3D, detecting social interactions, and verifying the compliance of recent safety measures due to the COVID-19 outbreak. We show that it is possible to rethink the concept of "social distancing" as a form of social interaction in contrast to a simple location-based rule. We publicly share the source code towards an open science mission.

Non-Pharmaceutical Interventions against COVID-19 Pandemic: Review of Contact Tracing and Social Distancing Technologies, Protocols, Apps, Security and Open Research Directions

“…Furthermore, a privacy-preserving computer vision-based social distancing framework was reported in [ 196 ]. This study aims at achieving a cost-effective social distancing framework that employs a neural network to detect humans using either fixed or mobile cameras and does not rely on ground plane estimation.…”

Vulnerable Road Users and Connected Autonomous Vehicles Interaction: A Survey

“…They are generally applied to model interaction between people or between people and environment. Some proposed works incorporate scene information in the predictive models, taking into consideration that trajectories remain within the driving environment [166,167].…”

Deep visual social distancing monitoring to combat COVID-19: A comprehensive survey

“…Put differently, people talking to each other strongly influence the risk of contagion more than walking apart. To that end, Bertoni et al (2021) develop a VSDM solution that analyzes SD based on both 3D localization and social cues. Typically, a DL-based VSDM method is proposed to detect people’s 3D locations and their body orientations from monocular cameras.…”

“… PETS2006 and real-world vdeo data Acc=95.2%, F1=97.5% Raise some privacy issues. ( Bertoni, Kreiss, & Alahi, 2021 ) DFCN A cost-effective VSD approach that perceives people’s 3D locations and their body orientation from images KITTI dataset Acc=84.7%, recall=85.3% Work with single RGB images, (ii) privacy safe, (iii) does not require homography calibration, (iii) generalize well across different datasets, (iv) work on fixed or moving cameras ( Rahim et al, 2021 ) YOLOv4 Validation on video data recordedc using fixed single motionless time of flight (ToF) camera ExDARK dataset mAP=97.84%, MAE=1.01 cm Can be applied in real-world scenarios because of high precision and the low error rate. Used only with fixed cameras.…”