Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo

Maurer, Daniel; Ju, Yong Chul; Breuß, Michael; Bruhn, Andrés

doi:10.1007/s11263-018-1079-1

Cited by 27 publications

(16 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [23] [24] use a winner-takes-all strategy to optimize the final disparity by choosing at each pixel the disparity associated with the minimal cost value. Other methods like graph cut [25] or coarse-to-fine refinement [26] are instead used for optimizing the final disparity. Besides the above methods computing cost volumes, methods based on statistical models, i.e.…”

Section: Related Work a Stereo Depth Estimationmentioning

confidence: 99%

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

Shi

Jiang

Guillemot

2019

IEEE Trans. on Image Process.

116

View full text Add to dashboard Cite

Light field is a type of image data that captures the 3D scene information by recording light rays emitted from a scene at various orientations. It offers a more immersive perception than classic 2D images but at the cost of huge data volume. In this paper, we draw inspiration from the visual characteristics of Sub-Aperture Images (SAIs) of light field and design a compact neural network representation for the light field compression task. The network backbone takes randomly initialized noise as input and is supervised on the SAIs of the target light field. It is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives. To further enhance compactness of the network meanwhile retain high quality of the decoded light field, we accordingly introduce modulator allocation and kernel tensor decomposition mechanisms, followed by non-uniform quantization and lossless entropy coding techniques, to finally form an efficient compression pipeline. Extensive experiments demonstrate that our method outperforms other state-of-the-art (SOTA) methods by a significant margin in the light field compression task. Moreover, after aligning descriptors, the modulators learned from one light field can be transferred to new light fields for rendering dense views, indicating a potential solution for view synthesis task.

show abstract

Section: Related Work a Stereo Depth Estimationmentioning

confidence: 99%

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

Shi

Jiang

Guillemot

2019

IEEE Trans. on Image Process.

116

View full text Add to dashboard Cite

show abstract

“…One could consider as loss function ρ the normalized sum of squared deviations ρ SSD (x, y) = 1 m m c=1 (x c − y c ) 2 or a robust variant of it, and then linearize (2) using first-order Taylor expansion as in [7,12]. However, linearization requires small depth increments.…”

Section: Introductionmentioning

confidence: 99%

“…Other methods combining stereo and shading information under natural illumination have also recently been developed [10,11,17], but they only consider photometry as a way to refine an existing multi-view 3D-reconstruction, which remains the baseline of the process. We would rather like to follow an end-to-end joint approach, as for instance in the very recent work [12]. In comparison with [12], the approach presented in the next section avoids linearization of the fidelity term and is therefore slightly more generic, since any robust photo-consistency measure (including those based on non-differentiable or non-convex loss functions) can be considered.…”

Section: Introductionmentioning

confidence: 99%

“…We would rather like to follow an end-to-end joint approach, as for instance in the very recent work [12]. In comparison with [12], the approach presented in the next section avoids linearization of the fidelity term and is therefore slightly more generic, since any robust photo-consistency measure (including those based on non-differentiable or non-convex loss functions) can be considered.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Splitting-Based Algorithm for Multi-view Stereopsis of Textureless Objects

Mélou

Quéau

Castan³

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We put forward a simple, yet effective splitting strategy for multi-view stereopsis. It recasts the minimization of the classic photoconsistency + gradient regularization functional as a sequence of simple problems which can be solved efficiently. This framework is able to handle various photo-consistency measures and regularization terms, and can be used for instance to estimate either a minimal-surface or a shading-aware solution. The latter makes the proposed approach very effective for dealing with the well-known problem of textureless objects 3D-reconstruction.

show abstract

“…Therefore, the SFS-based reconstruction method has attracted attention from numerous researchers. Maurer et al [16] used a method that combined SFS and stereo to establish the functional energy with a detail-preserving anisotropic second-order smoothness term, and estimated the depth of the object surface. Lu et al [17] proposed a Lambert-Phong hybrid model algorithm, and obtained the coordinate information of the highlighted region of the droplet surface by the mask regions with convolutional neural network (R-CNN).…”

mentioning

confidence: 99%

Three-Dimensional Reconstruction Method for Machined Surface Topography Based on Gray Gradient Constraints

Shi

Zheng

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

Featured Application: Applying on mechanical engineering. Abstract: In the modern engineering field, recovering the machined surface topography is important for studying mechanical product function and surface characteristics by using the shape from shading (SFS)-based reconstruction method. However, due to the limitations of many constraints and oversmoothing, the existing SFS-based reconstruction methods are not suitable for machined surface topography. This paper presents a new three-dimensional (3D) reconstruction method of machined surface topography. By combining the basic principle of SFS and the analytic method, the analytic model of a surface gradient is established using the gray gradient as a constraint condition. By efficiently solving the effect of quantization errors and ambiguity of the gray scale on reconstruction accuracy using a wavelet denoising algorithm and image processing technology, the reconstruction algorithm is implemented for machined surface topography. Experimental results on synthetic images and machined surface topography images show that the proposed algorithm can accurately and efficiently recover the 3D shape of machined surface topography.Appl. Sci. 2019, 9, 591 2 of 16 wear particle surfaces based on a photometric stereo, and obtained the surface topographies of wear particles for further feature-based wear particle identification. Tang et al. [9] used the 3D reconstruction method based on the shape from a focus to determine the grinding wheel surface topography.Three-dimensional reconstruction is the process of recovering 3D shapes of objects from single or multiple images [10][11][12], and the shape from shading (SFS) method is considered to be one of the fastest and most efficient 3D reconstruction methods [13][14][15]. Therefore, the SFS-based reconstruction method has attracted attention from numerous researchers. Maurer et al. [16] used a method that combined SFS and stereo to establish the functional energy with a detail-preserving anisotropic second-order smoothness term, and estimated the depth of the object surface. Lu et al. [17] proposed a Lambert-Phong hybrid model algorithm, and obtained the coordinate information of the highlighted region of the droplet surface by the mask regions with convolutional neural network (R-CNN). Furthermore, they used the Taylor expansion and Newton iteration method in the reflection model to obtain the heights of all the positions. Sun et al. [18] proposed a new roughness evaluation method based on the complex wavelet enhanced SFS transform, and recovered the 3D topography of a milling surface using the 3D models that were developed from the digital image. Zhu et al. [19] proposed a novel method that uses Fourier transform to fuse the depth from focus (DFF) and SFS, and achieved the reconstruction of the tool wear image in addition to ensuring the continuity of the depth map. Yang et al. [20] used the SFS algorithm to reconstruct the 3D shapes of the welding seam, and extracted the curvature information as the feature vector of the ...

show abstract

Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth, Illumination and Albedo

Cited by 27 publications

References 56 publications

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

A Framework for Learning Depth From a Flexible Subset of Dense and Sparse Light Field Views

A Splitting-Based Algorithm for Multi-view Stereopsis of Textureless Objects

Three-Dimensional Reconstruction Method for Machined Surface Topography Based on Gray Gradient Constraints

Contact Info

Product

Resources

About