Microsoft Azure Kinect Calibration for Three-Dimensional Dense Point Clouds and Reliable Skeletons

Abstract

Nowadays, the need for reliable and low-cost multi-camera systems is increasing for many potential applications, such as localization and mapping, human activity recognition, hand and gesture analysis, and object detection and localization. However, a precise camera calibration approach is mandatory for enabling further applications that require high precision. This paper analyzes the available two-camera calibration approaches to propose a guideline for calibrating multiple Azure Kinect RGB-D sensors to achieve the best alignment of point clouds in both color and infrared resolutions, and skeletal joints returned by the Microsoft Azure Body Tracking library. Different calibration methodologies using 2D and 3D approaches, all exploiting the functionalities within the Azure Kinect devices, are presented. Experiments demonstrate that the best results are returned by applying 3D calibration procedures, which give an average distance between all couples of corresponding points of point clouds in color or an infrared resolution of 21.426 mm and 9.872 mm for a static experiment and of 20.868 mm and 7.429 mm while framing a dynamic scene. At the same time, the best results in body joint alignment are achieved by three-dimensional procedures on images captured by the infrared sensors, resulting in an average error of 35.410 mm.

Simulation of Human Movement in Zero Gravity

“… Investigate new algorithms, like those discussed by Eggert et al [ 28 ], for transforming the coordinate systems of motion captures to address occasional failures in Arun’s method [ 21 ]. Alternatively, consider adopting one of the reliable calibration routines proposed by Romeo et al [ 29 ]. Improve the merging of captures by introducing a new measure for judging the consistency of tracked motion curves, beyond relying solely on confidence level indicators.…”

“…Investigate new algorithms, like those discussed by Eggert et al [ 28 ], for transforming the coordinate systems of motion captures to address occasional failures in Arun’s method [ 21 ]. Alternatively, consider adopting one of the reliable calibration routines proposed by Romeo et al [ 29 ].…”