Describing Textures in the Wild

Abstract

Patterns and textures are defining characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly. Aiming at supporting this analytical dimension in image understanding, we address the challenging problem of describing textures with semantic attributes. We identify a rich vocabulary of forty-seven texture terms and use them to describe a large dataset of patterns collected "in the wild". The resulting Describable Textures Dataset (DTD) is the basis to seek for the best texture representation for recognizing describable texture attributes in images. We port from object recognition to texture recognition the Improved Fisher Vector (IFV) and show that, surprisingly, it outperforms specialized texture descriptors not only on our problem, but also in established material recognition datasets. We also show that the describable attributes are excellent texture descriptors, transferring between datasets and tasks; in particular, combined with IFV, they significantly outperform the state-of-the-art by more than 8% on both FMD and KTH-TIPS-2b benchmarks. We also demonstrate that they produce intuitive descriptions of materials and Internet images.

Using filter banks in Convolutional Neural Networks for texture classification

“…We reproduce the setup of [3] in which 92 images are used per class with 46 for training, 46 for testing. We resize the 200x200 images to 227x227.…”

“…DTD [3] contains 47 classes of 120 images "in the wild" each. The images are of various sizes and even though using multiple input sizes should be possible with our T-CNN, we resize all the images to 227x227 for comparison with AlexNet which requires fixed input images.…”

“…ImNet-S2 [14] 99.8 ±0.1 [3] 75.5 ±0.8 [4] 81.5 ±2.0 [4] ---containing object-like images. This is due to the fact that the first convolution layer extracts extremely simple features (mainly edges) and acts like a Gabor filter which is not robust at classifying object datasets.…”

Rotation invariant texture descriptors based on Gaussian Markov random fields for classification

“…3 illustrates that RI-LPH descriptors perform better than I-LPH descriptors. method dataset BRODATZ CURET LBP (Ojala et al, 2002) 97.1 93.3 SH (Liu and Wang, 2003) 84.6 86.4 LBP HF (Zhao et al, 2012) 97.4 90.6 PRI-CoLBP (Qi et al, 2012) 96.6 99.2 DeCAF (Cimpoi et al, 2014) 97.9 RI-LPH 98.0 95.6 I-LPH 94.4 89.4…”

“…Texture analysis has been extensively studied in recent years and a large number of texture feature extraction techniques have been developed (Cohen et al, 1991;Nixon and Aguado, 2008;Varma and Zisserman, 2009;Zhao et al, 2012;Chen et al, 2010;Liu and Fieguth, 2012;Lei et al, 2014;Qi et al, 2012;Hinton and Salakhutdinov, 2006;Cimpoi et al, 2014;Simonyan et al, 2014). These methods can be roughly grouped into four main categories, namely statistical, structural, spectral and model based feature extraction methods (Xie and Mirmehdi, 2008).…”