Differential Angular Imaging for Material Recognition

Abstract

Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation "in the wild." Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose a middle-ground approach that takes advantage of both rich radiometric cues and flexible image capture. We develop a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS), geared towards real use for autonomous agents. This publicly available database 1 consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.

Deep Texture Manifold for Ground Terrain Recognition

“…The size of material datasets have also increased from roughly 100 images in each class [4,35] to over 1000 images in each class [1,36]. The Ground Terrain in Outdoor Scenes (GTOS) dataset has been used with angular differential imaging [36] for material recognition based on angular gradients. For our work, single images are used for recognition without variation in viewing direction, so reflectance gradients are not considered.…”

“…The training set is a ground terrain database (GTOS) [36] with 31 classes of ground terrain images (over 30,000 images in the dataset). Instead of using images from the GTOS dataset for testing, we collect GTOS-mobile, 81 ground terrains videos of similar terrain classes captured with a handheld mobile phone and with arbitrary lighting/viewpoint.…”

PhotoShape

“…the Flickr Material Database [Sharan et al 2014] contains 100 images per class, with the images not beeing representative of everyday scenes). Another dataset for surface material recognition is presented in [Xue et al 2017], together with a classification network based on differentiable angular imaging. Moreover, [Wang et al 2016b] introduces a lightfield dataset for material recognition.…”

DEFO-NET: Learning Body Deformation Using Generative Adversarial Networks

“…The segmented body can then be upsampled to form a 3-D voxel representation of the object to be traversed. In addition, the material of the body can also be supplied as one of the conditional parameters and obtained from different methods, i.e., the recent Differential Angular Imaging for Material Recognition (DAIN) network [7], trained on the GTOS (Ground Terrain in Outdoor Scenes) material reflectance database composed of 40 surface classes. A prediction on a single Titan X GPU takes under a second, orders of magnitude faster than an FEM simulator.…”

“…In [5] the authors recently proposed a method for pre-computation of the dymanics of fluid spaces using implicit surfaces. Material recognition is another problem that has been investigated using Convolutional Neural Networks [6] [7].…”