OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks

Abstract

OpenPTrack is an open source software for multi-camera calibration and people tracking in RGB-D camera networks. It allows to track people in big volumes at sensor frame rate and currently supports a heterogeneous set of 3D sensors. In this work, we describe its user-friendly calibration procedure, which consists of simple steps with real-time feedback that allow to obtain accurate results in estimating the camera poses that are then used for tracking people. On top of a calibration based on moving a checkerboard within the tracking space and on a global optimization of cameras and checkerboards poses, a novel procedure which aligns people detections coming from all sensors in a x-y-time space is used for refining camera poses. While people detection is executed locally, in the machines connected to each sensor, tracking is performed by a single node which takes into account detections from all over the network. Here we detail how a cascade of algorithms working on depth point clouds and color, infrared and disparity images is used to perform people detection from different types of sensors and in any indoor light condition. We present experiments showing that a considerable improvement can be obtained with the proposed calibration refinement procedure that exploits people detections and we compare Kinect v1, Kinect v2 and Mesa SR4500 performance for people tracking applications. OpenPTrack is based on the Robot Operating System and the Point Cloud Library and has already been adopted in networks composed of up to ten imagers for interactive arts, education, culture and human-robot interaction applications.

An Inverse Perspective Mapping Approach using Monocular Camera of Pepper Humanoid Robot to Determine the Position of Other Moving Robot in Plane

“…This is to minimize the ease of knowing the wrong positives caused by other things within the Pepper frame. In previous steps, the pixel points of the potential object measured by the image are known, but to calculate it according to the global framework, the calibration tool should be a reasonable unit of measurement (i.e there is a lot of calibration methods, such as (Boyle et al, 2000) (Munaro et al, 2016). The methods like chessboard calibration and marking points calibration discussed in (Miraldo and Araujo, 2013), (Moreno and Taubin, 2012) can be used as well but in this case, the proposed system would be implemented with a Pepper camera in a fixed or moving position.…”

Comprehensive Analysis of System Calibration between Optical Camera and Range Finder

“…Their two-step system calibration produced several pairs of ROPs, and they selected the best one which minimizes the re-projection error for all image pairs. Mattero et al [26] generated a multi-modal sensor system for people tracking, and the system is based on multi-camera network composed by three Kinect sensors and three SR4500 TOF cameras. They utilized OpenPTrack software in the calibration steps.…”

“…However, some of the studies simply used nominal values of IOPs provided by the sensor manufacturer in their system calibration procedures, without any correction [17]. Also, other studies did not consider some IOPs such as range distortion parameters ( [17,21,[24][25][26]) and focal length ( [3]) in their self-calibration procedures. To minimize the effect of the self-calibration quality on the accuracy of system calibration and also the quality of RGB-D data produced, the proposed study considers all the involved sensors and their IOPs.…”

FusionKit

“…In latter case, moving objects or users are tracked using particle models and a depth-aware optical flow algorithm. OpenPTrack [17] is an open source software to track people using multiple networked depth sensors e.g. for surveillance scenarios.…”