Deep Photometric Stereo Network

Santo, Hiroaki; Samejima, Masaki; Sugano, Yusuke; Shi, Boxin; Matsushita, Yasuyuki

doi:10.1109/iccvw.2017.66

Cited by 120 publications

(117 citation statements)

References 22 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…Machine learning techniques have been applied in a few very recent photometric stereo works [21,19]. Santo et al [19] presented a supervised learningbased photometric stereo method using a neural network that takes as input a normalized vector where each element corresponds to an observation under specific illumination. A surface normal is predicted by feeding the vector to one dropout layer and adjacent six dense layers.…”

Section: Related Workmentioning

confidence: 99%

CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces

Ikehata

2018

Lecture Notes in Computer Science

121

267

View full text Add to dashboard Cite

Most conventional photometric stereo algorithms inversely solve a BRDF-based image formation model. However, the actual imaging process is often far more complex due to the global light transport on the non-convex surfaces. This paper presents a photometric stereo network that directly learns relationships between the photometric stereo input and surface normals of a scene. For handling unordered, arbitrary number of input images, we merge all the input data to the intermediate representation called observation map that has a fixed shape, is able to be fed into a CNN. To improve both training and prediction, we take into account the rotational pseudo-invariance of the observation map that is derived from the isotropic constraint. For training the network, we create a synthetic photometric stereo dataset that is generated by a physics-based renderer, therefore the global light transport is considered. Our experimental results on both synthetic and real datasets show that our method outperforms conventional BRDF-based photometric stereo algorithms especially when scenes are highly non-convex.

show abstract

Section: Related Workmentioning

confidence: 99%

CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces

Ikehata

2018

Lecture Notes in Computer Science

121

267

View full text Add to dashboard Cite

show abstract

“…Recently, Santo et al [8] proposed a deep fully-connected network, called DPSN, to regress per-pixel normal given a fixed number of observations (e.g., 96) captured under a pre-defined set of light directions. For each image point of the object, all its observations are concatenated to form a fixed-length vector, which is fed into a fully-connected network to regress a single normal vector.…”

Section: Related Workmentioning

confidence: 99%

“…For the sake of data loading efficiency, we stored our training data in 8-bit PNG format. Blobby dataset We first followed [8] to render our training data using the blobby shape dataset [10], which contains 10 blobby shapes with various normal distributions. For each blobby shape, 1, 296 regularly-sampled views (36 azimuth angles × 36 elevation angles) were used, and for each view, 2 out of 100 BRDFs were randomly selected, leading to 25,920 samples (10 × 36 × 36 × 2).…”

Section: Synthetic Data For Trainingmentioning

confidence: 99%

“…The black boxes in the ground-truth normal maps are regions with cast shadows. Our method can produce more robust estimations in those regions compared with DPSN [8].6.3 Benchmark comparisonsDiLiGenT benchmark main dataset We compared PS-FCN against the recently proposed learning based method DPSN[8] and other previous state-of-theart methods. Quantitative results on the main dataset of the DiLiGenT benchmark are shown in Tab.…”

mentioning

confidence: 99%

See 1 more Smart Citation

PS-FCN: A Flexible Learning Framework for Photometric Stereo

Chen

Han

Wong

2018

Lecture Notes in Computer Science

122

189

View full text Add to dashboard Cite

This paper addresses the problem of photometric stereo for non-Lambertian surfaces. Existing approaches often adopt simplified reflectance models to make the problem more tractable, but this greatly hinders their applications on real-world objects. In this paper, we propose a deep fully convolutional network, called PS-FCN, that takes an arbitrary number of images of a static object captured under different light directions with a fixed camera as input, and predicts a normal map of the object in a fast feed-forward pass. Unlike the recently proposed learning based method, PS-FCN does not require a pre-defined set of light directions during training and testing, and can handle multiple images and light directions in an order-agnostic manner. Although we train PS-FCN on synthetic data, it can generalize well on real datasets. We further show that PS-FCN can be easily extended to handle the problem of uncalibrated photometric stereo. Extensive experiments on public real datasets show that PS-FCN outperforms existing approaches in calibrated photometric stereo, and promising results are achieved in uncalibrated scenario, clearly demonstrating its effectiveness.

show abstract

“…Instead of explicit modeling image formation process and global illumination effects as in traditional methods, deep learning based methods attempt to learn such information from data. DPSN [24] is the first attempt and it uses a deep fully-connect network to regress surface normals from given observations captured under pre-defined lightings in a supervised manner. However, pre-definition of lightings limits its practicality for photometric stereo where the number of inputs often varies.…”

Section: Related Workmentioning

confidence: 99%

SPLINE-Net: Sparse Photometric Stereo Through Lighting Interpolation and Normal Estimation Networks

Zheng

Jia

Shi

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

This paper solves the Sparse Photometric stereo through Lighting Interpolation and Normal Estimation using a generative Network (SPLINE-Net). SPLINE-Net contains a lighting interpolation network to generate dense lighting observations given a sparse set of lights as inputs followed by a normal estimation network to estimate surface normals. Both networks are jointly constrained by the proposed symmetric and asymmetric loss functions to enforce isotropic constrain and perform outlier rejection of global illumination effects. SPLINE-Net is verified to outperform existing methods for photometric stereo of general BRDFs by using only ten images of different lights instead of using nearly one hundred images.

show abstract

Deep Photometric Stereo Network

Cited by 120 publications

References 22 publications

CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces

CNN-PS: CNN-Based Photometric Stereo for General Non-convex Surfaces

PS-FCN: A Flexible Learning Framework for Photometric Stereo

SPLINE-Net: Sparse Photometric Stereo Through Lighting Interpolation and Normal Estimation Networks

Contact Info

Product

Resources

About