Real-time human motion tracking using multiple depth cameras

Zhang, Licong; Sturm, Jürgen; Cremers, Daniel; Lee, Dongheui

doi:10.1109/iros.2012.6385968

Cited by 85 publications

(43 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of multiple cameras for tracking people has been demonstrated in previous research with overlapping [22][23][24] and non-overlapping [25,26] fields of view. A similar work is [27] in which multiple Kinect depth streams are combined, whereas we merge the skeleton streams while accounting for occlusion, arguably more useful for rapid design and prototyping of HCI systems.…”

Section: Related Workmentioning

confidence: 99%

Out of sight: a toolkit for tracking occluded human joint positions

Quigley

Harris-Birtill

2016

Pers Ubiquit Comput

View full text Add to dashboard Cite

Real-time identification and tracking of the joint positions of people can be achieved with off-the-shelf sensing technologies such as the Microsoft Kinect, or other camera-based systems with computer vision. However, tracking is constrained by the system's field of view of people. When a person is occluded from the camera view, their position can no longer be followed. Out of Sight addresses the occlusion problem in depth-sensing tracking systems. Our new tracking infrastructure provides human skeleton joint positions during occlusion, by combining the field of view of multiple Kinects using geometric calibration and affine transformation. We verified the technique's accuracy through a system evaluation consisting of 20 participants in stationary position and in motion, with two Kinects positioned parallel, 45, and 90 apart. Results show that our skeleton matching is accurate to within 16.1 cm (s.d. = 5.8 cm), which is within a person's personal space. In a realistic scenario study, groups of two people quickly occlude each other, and occlusion is resolved for 85% of the participants. A RESTful API was developed to allow distributed access of occlusion-free skeleton joint positions. As a further contribution, we provide the system as open source.

show abstract

Section: Related Workmentioning

confidence: 99%

Out of sight: a toolkit for tracking occluded human joint positions

Quigley

Harris-Birtill

2016

Pers Ubiquit Comput

View full text Add to dashboard Cite

show abstract

“…Low-cost depth cameras such as the Microsoft Kinect stimulated the development of novel approaches to people detection and tracking on depth images [14,30,19,35]. Shotton et al [14] learn deep decision trees to detect and track up to 20 body parts of the human body.…”

Section: Related Workmentioning

confidence: 99%

FollowMe: Person following and gesture recognition with a quadrocopter

Naseer

Sturm

Cremers

2013

2013 IEEE/RSJ International Conference on Intelligent Robots and Systems

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we present an approach that allows a quadrocopter to follow a person and to recognize simple gestures using an onboard depth camera. This enables novel applications such as hands-free filming and picture taking. The problem of tracking a person with an onboard camera however is highly challenging due to the self-motion of the platform. To overcome this problem, we stabilize the depth image by warping it to a virtual-static camera, using the estimated pose of the quadrocopter obtained from vision and inertial sensors using an Extended Kalman filter. We show that such a stabilized depth video is well suited to use with existing person trackers such as the OpenNI tracker. Using this approach, the quadrocopter not only obtains the position and orientation of the tracked person, but also the full body pose -which can then for example be used to recognize hand gestures to control the quadrocopter's behaviour. We implemented a small set of example commands ("follow me", "take picture", "land"), and generate corresponding motion commands. We demonstrate the practical performance of our approach in an extensive set of experiments with a quadrocopter. Although our current system is limited to indoor environments and small motions due to the restrictions of the used depth sensor, it indicates that there is large potential for such applications in the near future.

show abstract

“…Multiple calibrated RGB-D cameras have been used in a wide variety of applications such as 3D reconstruction [1][2][3], people tracking [4][5][6], motion capture [7][8][9], object tracking [10], augmented reality [11][12][13], and telepresence [14][15][16] among others.…”

Section: Introductionmentioning

confidence: 99%

Multiple Kinect V2 Calibration

Córdova-Esparza

Terven²,

Hernandez

et al. 2016

Automatika

View full text Add to dashboard Cite

In this paper, we propose a method to easily calibrate multiple Kinect V2 sensors. It requires the cameras to simultaneously observe a 1D object shown at different orientations (three at least) or a 2D object for at least one acquisition. This is possible due to the built-in coordinate mapping capabilities of the Kinect. Our method follows five steps: image acquisition, pre-calibration, point cloud matching, intrinsic parameters initialization, and final calibration. We modeled radial and distortion parameters of all the cameras, obtaining a root mean square reprojection error of 0.2 pixels on the depth cameras and 0.4 pixels on the color cameras. To validate the calibration results we performed point cloud fusion with color and 3D reconstruction using the depth and color information from four Kinect sensors.Key words: Camera calibration, Kinect V2 system, Multiple cameras Umjeravanje Kinect V2 sustava s više kamera. U ovom je radu predložena metoda za jednostavno umjeravanje proizvoljnog broja senzora Kinect V2. Izvodi se istovremenim snimanjem objekta s više kamera. Jednodimenzionalan objekt potrebno je snimiti s najmanje 3 različite orijentacije, a dvodimenzionalan s najmanje jedne orijentacije. Istovremeno snimanje s više kamera moguće je zahvaljujući integriranom mapiranju koordinata u Kinect sustavu. Predložena metoda izvodi se u pet koraka: akvizicija slike, predumjeravanje, uskladivanje oblaka točaka, inicijalizacija intrinzičnih parametara i konačno umjeravanje. U radu su modelirani radijalni i distorzijski parametri svih kamera, pričemu se ostvaruje korijen srednje kvadratične pogreške ponovne projekcije iznosa 0:2 piksela na kamerama dubine i 0:4 piksela na kamerama u boji. Za validaciju rezultata umjeravanja provedena je fuzija oblaka točaka s rekonstrukcijom trodimenzionalnog objekta i boje korištenjem informacije o dubini i boji š cetiri Kinect senzora.

show abstract

Real-time human motion tracking using multiple depth cameras

Cited by 85 publications

References 18 publications

Out of sight: a toolkit for tracking occluded human joint positions

Out of sight: a toolkit for tracking occluded human joint positions

FollowMe: Person following and gesture recognition with a quadrocopter

Multiple Kinect V2 Calibration

Contact Info

Product

Resources

About