Super-Fine Attributes with Crowd Prototyping

Abstract

Recognising human attributes from surveillance footage is widely studied for attribute-based re-identification. However, most works assume coarse, expertly-defined categories, ineffective in describing challenging images. Such brittle representations are limited in descriminitive power and hamper the efficacy of learnt estimators. We aim to discover more relevant and precise subject descriptions, improving image retrieval and closing the semantic gap. Inspired by fine-grained and relative attributes, we introduce super-fine attributes, which now describe multiple, integral concepts of a single trait as multi-dimensional perceptual coordinates. Crowd prototyping facilitates efficient crowdsourcing of super-fine labels by pre-discovering salient perceptual concepts for prototype matching. We re-annotate gender, age and ethnicity traits from PETA, a highly diverse (19K instances, 8.7K identities) amalgamation of 10 re-id datasets including VIPER, CUHK and TownCentre. Employing joint attribute regression with the ResNet-152 CNN, we demonstrate substantially improved ranked retrieval performance with super-fine attributes in direct comparison to conventional binary labels, reporting up to a 11.2% and 14.8% mAP improvement for gender and age, further surpassed by ethnicity. We also find our 3 super-fine traits to outperform 35 binary attributes by 6.5% mAP for subject retrieval in a challenging zero-shot identification scenario.

Transfer Learning Based Approach for Semantic Person Retrieval

“…Increasingly, deep learning is used for semantic person identification [6,7,8], by virtue of performance. [8] used the hand-crafted features and Extra Tree Classification (ETC) algorithm for identification while [7] used a Convolution Neural Network based approach named Semantic Retrieval Convolution Neural Network (SRCNN).…”

“…However, these approaches suffered from the small number of training data, hence 3.9 % at rank 1 on zero-shot identification. To overcome this problem, [6] applied fine-tuned ResNet-152 which had been pre-trained on ILSVRC 2012 dataset [10] for attribute classification. There might still be room for improvement by using a newer classification algorithm.…”

“…suggested the use of transfer learning model [12]. For person identification, [6] applied fine-tuned ResNet-151 model pre-trained on ImageNet Large Scale Visual Recognition Competition (ILSVRC) 2012 dataset [10]. Motivated by these approaches, we develop the transfer learning based approach for semantic person identification and search.…”

Feature Selection for Subject Ranking using Soft Biometric Queries

“…The relative importance of the individual features has been a subject of interest in many papers ( [10], [14], [15], [16]). For instance, in [10] it is remarked that weighting features based on their distinctiveness and permanence can have a high positive impact on the results.…”

“…Finding such internal correlations would be useful not only to reduce the number of features to a smaller relevant set, but also to infer the value of missing features in cases they cannot be extracted properly. In [14] the main focus was the usage of relative rather than absolute features. The work concluded that relative features clearly outperformed the categorical ones because of their ability to assign soft probabilities.…”