Color image enhancement using depth and intensity measurements of a time-of-flight depth camera

Abstract

A time-of-flight (ToF) depth camera can capture a depth map of the scene by measuring the phase delay between the emitted and reflected infrared (IR) light signals. In addition, an intensity map that represents the magnitude of the reflected light can be obtained by the ToF camera. If we consider the light source of the ToF camera as a flash, the intensity map can be deemed as an IR flashed image. Building on ideas from flash/no-flash photography and dark flash photography, we devise a color image enhancement framework that exploits information from the intensity and depth maps. To this end, ToF-related distortions of the intensity and depth maps are first reduced. We then restore fine details of color images captured under weak illumination by combining mutually beneficial information from the visible and IR band signals. In addition, we show that the depth map can be used to produce depth-adaptive effects such as depth-adaptive smoothing at the resultant color image.