Single Image Intrinsic Decomposition Without a Single Intrinsic Image

The Sound of Pixels

“…Self-supervised learning. Our work builds off efforts to learn perceptual models that are "self-supervised" by leveraging natural contextual signals in images [10,22,33,38,24], videos [46,32,43,44,13,20], and even radio signals [48]. These approaches utilize the power of supervised learning while not requiring manual annotations, instead deriving supervisory signals from the structure in Procedure to generate the sound of a pixel: pixel-level visual features are extracted by temporal max-pooling over the output of a dilated ResNet applied to T frames.…”

Leveraging Multi-View Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation

“…Most recently, unsupervised methods have been presented in which the training is performed on image sequences taken from fixed-position, time-lapse video with varying illumination (Li and Snavely 2018b;Ma et al 2018). In these networks, a major source of guidance for unsupervised training is the temporal consistency of reflectance for static regions within a sequence.…”

“…Our proposed system also trains on multiple images in an unsupervised manner and can be applied at test time on single images. Different from the previous fixed-view multi-image techniques (Li and Snavely 2018b;Ma et al 2018), our network uses unconstrained multi-view images and deals specifically with misalignment issues that arise in this setting. Such image sequences from unconstrained random views are much easier to obtain than fixed-view images.…”

“…Related observations have been made in other recent works. For example, Ma et al (Ma et al 2018) mention that their method cannot handle specularity well, and this limitation will be addressed in future work. Also, Shi et al (Shi et al 2017) discuss that the multiplicative intrinsic image decomposition model, I d = A • S, cannot adequately account for additive highlight components.…”

“…However, obtaining large-scale ground-truth data for training deep neural networks remains a challenge, and this has motivated recent work on developing unsupervised schemes for the reflectance separation problem. The unsupervised techniques that have been presented thus far all take the same approach of training a network on image sequences of a fixed scene under changing illumination (Li and Snavely 2018b;Ma et al 2018). With images from such a sequence, these methods guide network training by exploiting the albedo consistency that exists for each scene point throughout the sequence.…”