Colour Helmholtz Stereopsis for Reconstruction of Complex Dynamic Scenes

Abstract

Abstract-Helmholtz Stereopsis (HS) is a powerful technique for reconstruction of scenes with arbitrary reflectance properties. However, previous formulations have been limited to static objects due to the requirement to sequentially capture reciprocal image pairs (i.e. two images with the camera and light source positions mutually interchanged). In this paper, we propose colour HS -a novel variant of the technique based on wavelength multiplexing. To address the new set of challenges introduced by multispectral data acquisition, the proposed novel pipeline for colour HS uniquely combines a tailored photometric calibration for multiple camera/light source pairs, a novel procedure for surface chromaticity calibration and the state-of-the-art Bayesian HS suitable for reconstruction from a minimal number of reciprocal pairs. Experimental results including quantitative and qualitative evaluation demonstrate that the method is suitable for flexible (single-shot) reconstruction of static scenes and reconstruction of dynamic scenes with complex surface reflectance properties.

Color Photometric Stereo Using Multi-Band Camera Constrained by Median Filter and Occluding Boundary

“…Figure 32a is smoother than Figure 32c, and Figure 32b is smoother than Figure 32d, since the albedo is not smooth, which means that the surface normal becomes relatively smooth. Although Figures 17,20,23,26, and 29 show over-smoothed result of albedo, it is an adequate way to smooth the albedo in order to obtain sharp features of surface normal. As shown in Figure 14, our method is robust to multiple types of albedos.…”

“…The color photometric stereo method [ 3 , 4 , 5 ] (also known as shape-from-color) was developed in the 1990s. Since then, various studies [ 4 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ] have been conducted in this regard. However, many problems are inherent in the color photometric stereo method.…”

“…Recently, various techniques have been proposed to apply the color photometric stereo method to multicolored objects. Roubtsova et al [ 20 ] applied the color photometric stereo method to objects with arbitrary BRDF (bidirectional reflectance distribution function) by incorporating the Helmholtz Stereo method. However, the principle of this method does not allow for real-time measurement.…”

“…The techniques of Gotardo et al [ 23 ], Kim et al [ 16 ], and Roubtsova et al [ 20 ] need to employ optical flow to measure a dynamic object, while the technique of Fyffe et al [ 12 ] requires a reflectance database to be prepared prior to the measurement. Our proposed technique does not require a shape obtained from other sensors such as multi-view stereo or laser sensor, unlike the technique of Anderson et al [ 6 ] Moreover, unlike the techniques of Chakrabarti et al [ 8 ] and Jiao et al [ 15 ] our proposed method does not require region segmentation.…”