Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields

Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon; Lin, Stephen

doi:10.1007/978-3-319-46484-8_9

Cited by 89 publications

(52 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…albedo shading average albedo shading average albedo shading average image split JCNF [15] 0.0070 0.0090 0.0080 0.0060 0.0070 0.0065 0.0920 0.1010 0.0970 Ours 0.0040 0.0052 0.0046 0.0030 0.0040 0.0035 0.1081 0.0815 0.0948 Table 3. Quantitative comparison on the auxilliary MPI-Sintel benchmark.…”

Section: Mse Lmse Dssimmentioning

confidence: 99%

Revisiting Deep Intrinsic Image Decompositions

Fan

Yang

Hua

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

110

135

View full text Add to dashboard Cite

While invaluable for many computer vision applications, decomposing a natural image into intrinsic reflectance and shading layers represents a challenging, underdetermined inverse problem. As opposed to strict reliance on conventional optimization or filtering solutions with strong prior assumptions, deep learning based approaches have also been proposed to compute intrinsic image decompositions when granted access to sufficient labeled training data. The downside is that current data sources are quite limited, and broadly speaking fall into one of two categories: either dense fully-labeled images in synthetic/narrow settings, or weakly-labeled data from relatively diverse natural scenes. In contrast to many previous learning-based approaches, which are often tailored to the structure of a particular dataset (and may not work well on others), we adopt core network structures that universally reflect loose prior knowledge regarding the intrinsic image formation process and can be largely shared across datasets. We then apply flexibly supervised loss layers that are customized for each source of ground truth labels. The resulting deep architecture achieves state-of-the-art results on all of the major intrinsic image benchmarks, and runs considerably faster than most at test time.

show abstract

Section: Mse Lmse Dssimmentioning

confidence: 99%

Revisiting Deep Intrinsic Image Decompositions

Fan

Yang

Hua

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

110

135

View full text Add to dashboard Cite

show abstract

“…Their training data is generated in a physics-based manner as well, including a specular component, but they do not explicitly embed a physics-based image formation loss. Another recent work [18] uses an image formation component in their unary term for CRF (for the optimization process, not in the learning process itself), but their training data (Sintel) was not created in a physics-based manner. Nonetheless, none of proposed deep learning methods consider the image formation process for consistent decomposition during training, nor a Retinex driven gradient separation approach [4,13,15,32,37,38].…”

Section: Supervised Deep Learningmentioning

confidence: 99%

CNN Based Learning Using Reflection and Retinex Models for Intrinsic Image Decomposition

Baslamisli

Gevers

2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

Most of the traditional work on intrinsic image decomposition rely on deriving priors about scene characteristics. On the other hand, recent research use deep learning models as in-and-out black box and do not consider the wellestablished, traditional image formation process as the basis of their intrinsic learning process. As a consequence, although current deep learning approaches show superior performance when considering quantitative benchmark results, traditional approaches are still dominant in achieving high qualitative results.In this paper, the aim is to exploit the best of the two worlds. A method is proposed that (1) is empowered by deep learning capabilities, (2) considers a physics-based reflection model to steer the learning process, and (3) exploits the traditional approach to obtain intrinsic images by exploiting reflectance and shading gradient information. The proposed model is fast to compute and allows for the integration of all intrinsic components. To train the new model, an object centered large-scale datasets with intrinsic ground-truth images are created.The evaluation results demonstrate that the new model outperforms existing methods. Visual inspection shows that the image formation loss function augments color reproduction and the use of gradient information produces sharper edges.Datasets, models and higher resolution images are available at https://ivi.fnwi.uva.nl/cv/retinet.

show abstract

“…For intrinsic image decomposition, [35] introduces the first unified model for recovering shape, reflectance, and chromatic illumination in a joint optimization framework. Other works [36,37], jointly predict depth and intrinsic property. Finally, [38] exploits the relation between the intrinsic property and objects (i.e.…”

Section: Related Workmentioning

confidence: 99%

Joint Learning of Intrinsic Images and Semantic Segmentation

Baslamisli

Groenestege

Das

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Semantic segmentation of outdoor scenes is problematic when there are variations in imaging conditions. It is known that albedo (reflectance) is invariant to all kinds of illumination effects. Thus, using reflectance images for semantic segmentation task can be favorable. Additionally, not only segmentation may benefit from reflectance, but also segmentation may be useful for reflectance computation. Therefore, in this paper, the tasks of semantic segmentation and intrinsic image decomposition are considered as a combined process by exploring their mutual relationship in a joint fashion. To that end, we propose a supervised end-to-end CNN architecture to jointly learn intrinsic image decomposition and semantic segmentation. We analyze the gains of addressing those two problems jointly. Moreover, new cascade CNN architectures for intrinsic-for-segmentation and segmentation-for-intrinsic are proposed as single tasks. Furthermore, a dataset of 35K synthetic images of natural environments is created with corresponding albedo and shading (intrinsics), as well as semantic labels (segmentation) assigned to each object/scene. The experiments show that joint learning of intrinsic image decomposition and semantic segmentation is beneficial for both tasks for natural scenes. Dataset and models are available at: https://ivi.fnwi.uva.nl/cv/intrinseg.

show abstract

Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields

Cited by 89 publications

References 39 publications

Revisiting Deep Intrinsic Image Decompositions

Revisiting Deep Intrinsic Image Decompositions

CNN Based Learning Using Reflection and Retinex Models for Intrinsic Image Decomposition

Joint Learning of Intrinsic Images and Semantic Segmentation

Contact Info

Product

Resources

About