Fast RGB-D people tracking for service robots

Abstract

Service robots have to robustly follow and interact with humans. In this paper, we propose a very fast multi-people tracking algorithm designed to be applied on mobile service robots. Our approach exploits RGB-D data and can run in real-time at very high frame rate on a standard laptop without the need for a GPU implementation. It also features a novel depth-based sub-clustering method which allows to detect people within groups or even standing near walls. Moreover, for limiting drifts and track ID switches, an online learning appearance classifier is proposed featuring a three-term joint likelihood. We compared the performances of our system with a number of state-of-the-art tracking algorithms on two public datasets acquired with three static Kinects and a moving stereo pair, respectively. In order to validate the 3D accuracy of our system, we created a new dataset in which RGB-D data are acquired by a moving robot. We made publicly available this dataset which is not only annotated by hand, but the ground-truth position of people and robot are acquired with a motion capture system in order to evaluate tracking accuracy and precision in 3D coordinates. Results of experiments on these datasets are presented, showing that, even without the need for a GPU, our approach achieves state-of-the-art accuracy and superior speed

Person tracking and following with 2D laser scanners

“…Tracks can be initiated in the simplest case by starting a new track for every unassigned measurement at each timestep [39,52]. A more conservative approach, which is useful in situations where noisy background measurements can arise, is to first start a potential track for every unassigned measurement, then to only intiate a confirmed track once the potential track has been assigned measurements across multiple timesteps [29].…”

“…Munaro et al [52] proposed Kobilarov et al [37] developed a person-following Segway robot using an omnidirectional camera and a laser scanner. Nonetheless, variable environment lighting, backgrounds and appearances of people are factors which can be difficult to control for and can mislead vision-reliant tracking systems.…”

Out of sight: a toolkit for tracking occluded human joint positions

“…4) consists of a server running the tracking application and a number of client programs installed on each computer running a Kinect v2 sensor. 2 The clients send serialized Kinect BodyFrames (using the Kinect v2 SDK) to the server via HTTP POST. The server performs the initial calibration and provides a RESTful API for accessing occlusion-free body joint positions.…”

“…Automated human body tracking is the ability to identify and follow individuals in an environment, usually through human pose estimation and spatial recognition software. Inexpensive depth-sensing technologies, such as the timeof-flight camera within the Microsoft Kinect, have enabled the human body to be segmented, and subsequently tracked, in systems such as pedestrian behavior analysis [1], human-robot interactions [2], gait recognition [3], and cross-device interactions [4]. It's common for researchers and developers to leverage such noninvasive tracking infrastructure (e.g., through the Microsoft Kinect Software Development Kit 1 ) to support gesture control and novel forms of human-computer interaction (HCI).…”

Designing a Video Monitoring System for AAL applications: the SPHERE Case Study

“…Nevertheless, the proposed modular software system allows us to plug and play different detectors and trackers. Other real time detectors based on colour and depth data, such as [17] and real time trackers [17,5], have been tested and tuned into the proposed system.…”