DARN: A Deep Adversarial Residual Network for Intrinsic Image Decomposition

Abstract

We present a new deep supervised learning method for intrinsic decomposition of a single image into its albedo and shading components. Our contributions are based on a new fully convolutional neural network that estimates absolute albedo and shading jointly. Our solution relies on a single end-to-end deep sequence of residual blocks and a perceptually-motivated metric formed by two adversarially trained discriminators.As opposed to classical intrinsic image decomposition work, it is fully data-driven, hence does not require any physical priors like shading smoothness or albedo sparsity, nor does it rely on geometric information such as depth. Compared to recent deep learning techniques, we simplify the architecture, making it easier to build and train, and constrain it to generate a valid and reversible decomposition. We rediscuss and augment the set of quantitative metrics so as to account for the more challenging recovery of non scale-invariant quantities.We train and demonstrate our architecture on the publicly available MPI Sintel dataset and its intrinsic image decomposition, show attenuated overfitting issues and discuss generalizability to other data. Results show that our work outperforms the state of the art deep algorithms both on the qualitative and quantitative aspect.

Joint Learning of Intrinsic Images and Semantic Segmentation

“…In addition, [15] proposes a model by introducing inter-links between decoder modules, based on the expectation that intrinsic components are correlated. Moreover, [16] demonstrates the capability of generative adversarial networks for the task. On the other hand, in more recent work, [17] considers an image formation loss together with gradient supervision to steer the learning process to achieve more vivid colors and sharper edges.…”

“…Baseline network architecture: The encoder part is composed of 6 convolution blocks with 3 × 3 kernels and stride of 2 having[16,32, 64, 128, 256, 256] feature maps. The encoder part is mirrored to build the decoder.…”

Neural Inverse Rendering of an Indoor Scene From a Single Image

“…Intrinsic image decomposition is a sub-problem of inverse rendering, where a single image is decomposed into albedo and shading. Recent methods learn intrinsic image decomposition from labeled synthetic data [17,26,34] and from unlabeled [20] or partially labeled real data [49,19,28,2]. Intrinsic image decomposition methods do not explicitly recover geometry or illumination but rather combine them together as shading.…”

Intrinsic Image Transformation via Scale Space Decomposition

“…Direct Intrinsics [37] is a successful early example of this type, using a (back then seemingly bold) multilayer CNN architecture to transform an image directly into shading and albedo. Successive models include the work of Kim et al [25] that predicts depth and the other intrinsic components together with a joint convolutional network that has shared intermediate layers and a joint CRF loss, and the DARN [32] network that incorporates a discriminator network and the adversarial training scheme to enhance of the performance of a "generator" network that produces the decomposition results. Image scale space and pyramid structures: The investigation of image scale space is no less old-fashioned than that of the intrinsic image decomposition in vision.…”

“…While models of similar ideas have been proposed (e.g. [37,32]), our model explores the scale space of the network input and output, and considers to simply the transformation as a whole by horizontally expanding the functor approximation pipeline into a parallel set of sub-band transformations.…”