Auto calibration of multi‐camera system for human pose estimation

Abstract

The multi-camera calibration is an essential step for many spatially aware applications, such as robotic navigation, augmented reality, and 3D human pose estimation. Traditional calibration methods use off-the-shelf checkerboards or triangles as the known world coordinate system and their corresponding corners are set as control points, which heavily depends on specific calibration patterns and is not suitable for calibration patterndenied environments. In this paper, an automatic calibration method is proposed to calibrate the multi-camera system without the aid of a known calibration pattern. The key idea of the proposed method is that the authors consider the human body, which is always available, as the counterpart of the calibration pattern. The authors' approach starts with binocular camera calibration, in which the extrinsic and intrinsic parameters are calculated in order and followed by a joint optimisation. With the results of each pair of binocular camera calibration, the multi-camera system calibration is carried out in three steps: (i) parameters initialisation, (ii) extrinsic parameters optimisation, and (iii) jointly optimising intrinsic and extrinsic parameters. Since the authors' approach does not require additional calibration patterns except for one visible person, it is flexible and easy to be implemented. Real experiments are conducted in different scenes, camera angles, and camera settings. Human pose estimation with the multi-camera system is additionally performed for exhaustive experiments. The experimental results demonstrate that the authors' method shows superior performance than the traditional method with the aid of a specific calibration pattern. | INTRODUCTIONCamera calibration is a fundamental requirement for the multicamera system, which has been widely used in many applications due to its low cost. Flexible and accurate camera calibration algorithm is the key to unlock the extensive deployment of a multi-camera system. Multi-camera calibration requires estimation of intrinsic parameters such as focal length and principal point of a single camera, as well as extrinsic parameters (i.e. rotation and translation) [1]. The intrinsic parameters are calibrated once on the factory floor, but these parameters may not be accurate enough for measurement, and they will be changed over time due to various factors such as heat, mechanical stress etc. Meanwhile, the positions between camera pairs would be changed when the multi-camera system is deployed in a new scene, and the extrinsic parameters need to be calibrated again. One of the important factors affecting the accuracy of the calibration is the number and distribution of common points in the captured scene. However, a multicamera system may share limited public fields of view [2]; thus, the major bottleneck of multi-camera calibration is the difficulty of detecting and matching the sufficient known common points from different cameras.There are many calibration methods based on specific calibration patterns whose elements include point...