People Tracking by Cooperative Fusion of RADAR and Camera Sensors

Abstract

Accurate 3D tracking of objects from monocular camera poses challenges due to the loss of depth during projection. Although ranging by RADAR has proven effective in highway environments, people tracking remains beyond the capability of single sensor systems. In this paper, we propose a cooperative RADAR-camera fusion method for people tracking on the ground plane. Using average person height, joint detection likelihood is calculated by back-projecting detections from the camera onto the RADAR Range-Azimuth data. Peaks in the joint likelihood, representing candidate targets, are fed into a Particle Filter tracker. Depending on the association outcome, particles are updated using the associated detections (Tracking by Detection), or by sampling the raw likelihood itself (Tracking Before Detection). Utilizing the raw likelihood data has the advantage that lost targets are continuously tracked even if the camera or RADAR signal is below the detection threshold. We show that in single target, uncluttered environments, the proposed method entirely outperforms camera-only tracking. Experiments in a real-world urban environment also confirm that the cooperative fusion tracker produces significantly better estimates, even in difficult and ambiguous situations.

A novel tracking system for human following robots with fusion of MMW radar and monocular vision

“…The Proposed method 0.12 m Real-time embedded system CA model by Cao S et al (Zhang and Cao, 2019) 0.29 m, 0.013 rad Real-time tracking with EKF PDA filter by Kim T et al (Kim and Song, 2016) 1.10 m Tracking of road barrier Linear Kalman filter with Compensated algorithm tracking by Jang et al (Jang et al, 2019) 0.63 m to 1.21 m depending on distance One front camera and four corner radars Particle filter By Dimitrievski M et al (Dimitrievski et al, 2019) 0.60 m to 1.22 m depending on distance Real-time tracking in urban environment multimodal data combination method is proposed. The fusion results help the robot overcome interference including illumination, occlusion and clustering.…”

“…Though most of them achieve satisfying result, these decision-level fusion methods rely much on accurate single-sensor detection and localization results. A similar recent work shows the tracking accuracy of the fusion plan (Dimitrievski et al, 2019), while the robustness to interference and its robot application potential is not validated.…”

CACTUS: Content-Aware Compression and Transmission Using Semantics for Automotive LiDAR Data

“…However, most data processing architectures are unable to elaborate efficiently a large volume of LiDAR data in a reasonable time [ 5 ]. In addition, sharing environmental information could be required in order to enable a more efficient scheduling of operation plans, reorganize navigation paths, or access more efficient classification tasks [ 6 , 7 ]. It is possible to overcome this bottleneck by employing some shared highly performing computational resources, moving most of the advanced processing toward cloud computing environments [ 8 , 9 , 10 , 11 ].…”

On the Angular Control of Rotating Lasers by Means of Line Calculus on Hyperboloids

“…The shortcomings of the calibration by matching a rigid physical device model have recently been admitted by leading scientists in the field [6]. The inaccuracies and instabilities inherent to the current calibration procedures are troublesome in applications where intrinsic localization, registration, and sensor fusion are involved [7]. Furthermore, last but not least, intrinsic calibration procedures based on a physical model cope with the determination of physical parameters that can rarely be measured directly, and are moreover rather virtual than physical, due to the idealised abstract nature of the model.…”