On categorising gender in surveillance imagery

Abstract

Categorising gender for soft biometric recognition is especially challenging from low quality surveillance footage. Our novel approach discovers super fine-grained visual taxonomies of gender from pairwise similarity comparisons, annotated via crowdsourcing. This paper presents our techniques for collection, interpretation and clustering of perceived visual similarities, and discusses the transition from pre-defined categorisation to similarity comparisons between subjects. We compare and evaluate our proposal on two diverse datasets, demonstrating the ability to describe multiple concepts, including ambiguity and uncertainty, that go beyond binary male-female designators. Our method is applicable to a wide range of soft biometric traits and image attributes, and can aid in efficiently annotating large-scale datasets, by generating more discriminative, reproducible and flexible categorical labels.

Super-Fine Attributes with Crowd Prototyping

“…This motivates us to find a versatile, super-fine solution that can discover and represent descriptions given any manner of trait. We extend the work of [14] that compared a relative attributes approach to a one-dimensional embedding of similarity comparisons, finding an almost identical ordering of gender for 100 visually clear images. In contrast, when applied to the more visually ambiguous and unclear images from PETA, the technique found a much less divisive split between male and female image classes.…”

“…On the other hand, [47] comprehensively investigates 5 types of similarity judgement, reporting that while triplet matching exhibits lower variance, pairwise ratings are less costly and also possess greater granularity. As super-fine attributes must embody higher descriptive power we employ continuous pairwise comparisons embedded with Metric Multidimensional Scaling (MDS) [48] building upon a preliminary study [14].…”

“…To capture the crowd's unconstrained perception of each trait, we propose a crowdsourcing task collecting n 2 pairwise similarity comparisons from a small subset of n = 95 subject images, as in [14]. In each question, respondents are asked to judge the appearance and visibility of a trait between two subject images, Fig.…”

“…We find binary measures ill-suited when describing challenging images, as in triplet and grid-based query solutions [44], [45]. We therefore elicit scaled responses with pairwise similarity comparisons, commonly used in judging perceptual distance between objects [1], [13], [55] and measuring qualitative properties without prior knowledge of their structure [14], [47]. It is widely acknowledged that counting the number of differences between objects is more objective than recalling their similarities, so we ask respondents to annotate intervals of dissimilarity.…”

“…We propose crowd prototyping to discover such visual prototypes and map the crowd's consensus of each trait as a perceptual space, prior to large-scale annotation. This involves crowdsourcing similarity comparisons from a subset of images as in [14] (Fig. 2a).…”