PS-FCN: A Flexible Learning Framework for Photometric Stereo

Abstract

This paper addresses the problem of photometric stereo for non-Lambertian surfaces. Existing approaches often adopt simplified reflectance models to make the problem more tractable, but this greatly hinders their applications on real-world objects. In this paper, we propose a deep fully convolutional network, called PS-FCN, that takes an arbitrary number of images of a static object captured under different light directions with a fixed camera as input, and predicts a normal map of the object in a fast feed-forward pass. Unlike the recently proposed learning based method, PS-FCN does not require a pre-defined set of light directions during training and testing, and can handle multiple images and light directions in an order-agnostic manner. Although we train PS-FCN on synthetic data, it can generalize well on real datasets. We further show that PS-FCN can be easily extended to handle the problem of uncalibrated photometric stereo. Extensive experiments on public real datasets show that PS-FCN outperforms existing approaches in calibrated photometric stereo, and promising results are achieved in uncalibrated scenario, clearly demonstrating its effectiveness.

SPLINE-Net: Sparse Photometric Stereo Through Lighting Interpolation and Normal Estimation Networks

“…However, pre-definition of lightings limits its practicality for photometric stereo where the number of inputs often varies. PS-FCN [7] is proposed to address such a limitation and handle images under various lightings in an order-agnostic manner by aggregating features of inputs using the max-pooling operation. CNN-PS [16] is another work to accept orderagnostic inputs by introducing observation map, which is a fixed shape representation invariant to inputs.…”

“…3 We re-train CNN-PS [16] by taking ten observed irradiance values as inputs to deal with the problem of photometric stereo using a small number of images. 4 3 We have conducted evaluations using the same training data and testing data for PS-FCN [7]. However, performance of PS-FCN [ is not as expected.…”

“…However, performance of PS-FCN [ is not as expected. The major reason is that PS-FCN [7] requires training data with various shapes while our training data CYCLESPS [16] only contains three shapes with diverse reflectance. For a fair comparison, we don't compare with PS-FCN [7] in our experiments.…”

“…The major reason is that PS-FCN [7] requires training data with various shapes while our training data CYCLESPS [16] only contains three shapes with diverse reflectance. For a fair comparison, we don't compare with PS-FCN [7] in our experiments. 4 Considering the overall quantitative results of CNN-PS [16] with default settings (taking 50 observed irradiance values as inputs) on CYCLESPS-TEST [16] (31.08 • and 34.90 • ) and DILIGENT [26] (14.10 • ), we report results re-trained by our setting.…”

“…These can be achieved by either exploring analytical Bidirectional Reflectance Distribution Function (BRDF) representations (e.g., [12]) and general BRDF properties (e.g., [28]) to model non-Lambertian interactions of lighting and surface normal or suppressing global effects by treating them as outliers (e.g., [35]). Recently, deep learning based approaches are introduced to solve these difficulties by implicitly learning both the image formation process and global illumination effects from training data (e.g., [7,16]). Figure 1.…”