Spatio-thermal depth correction of RGB-D sensors based on Gaussian processes in real-time

Abstract

Commodity RGB-D sensors capture color images along with dense pixel-wise depth information in real-time. Typical RGB-D sensors are provided with a factory calibration and exhibit erratic depth readings due to coarse calibration values, ageing and thermal influence effects. This limits their applicability in computer vision and robotics. We propose a novel method to accurately calibrate depth considering spatial and thermal influences jointly. Our work is based on Gaussian Process Regression in a four dimensional Cartesian and thermal domain. We propose to leverage modern GPUs for dense depth map correction in real-time. For reproducibility we make our dataset and source code publicly available.

A Method to Compensate for the Errors Caused by Temperature in Structured-Light 3D Cameras

“…Moreover, our distortion model only requires of three parameters which can be obtained using a simple least squares optimization. Because of this simplicity the correction computation cost is minimal ( s per depth map using CPU) while the other methods [ 34 ] require up to 20 s per depth map and also the use of GPUs to accelerate computations in order to obtain real time results. In addition, if the comparison is done in terms of adaptability, the proposed approach can be easily adapted to other cameras.…”

“…Its method is based on a temperature compensation model that collects dot pattern images for different ambient temperatures, applies a regression model for each dot position, and recovers a reference view of the pattern for any given ambient temperature before applying stereo searching algorithm for calculating depth information. More recently, Heindl et al [ 34 ] proposed a real-time pixel-wise depth correction method for RGB-D cameras considering both spatial and thermal aspects. The method is based on a Gaussian Process Regression in a four dimensional Cartesian and thermal domain.…”

“…Despite proposed solutions to correct thermal effects on some RGB-D cameras, there are limitations such as the closely guarded secret of proprietary solutions [ 33 ] or the required computation cost [ 34 ] that hinder the implementation of these solutions to other cameras. Therefore, in case of applications where used sensors do not compensate for the thermal error, specific solutions to mitigate or eliminate it have to be applied.…”