Towards automated caricature recognition

Abstract

This paper addresses the problem of identifying a subject from a caricature. A caricature is a facial sketch of a subject's face that exaggerates identifiable facial features beyond realism, while still conveying his identity. To enable this task, we propose a set of qualitative facial features that encodes the appearance of both caricatures and photographs. We utilized crowdsourcing, through Amazon's Mechanical Turk service, to assist in the labeling of the qualitative features. Using these features, we combine logistic regression, multiple kernel learning, and support vector machines to generate a similarity score between a caricature and a facial photograph. Experiments are conducted on a dataset of 196 pairs of caricatures and photographs, which we have made publicly available. Through the development of novel feature representations and matching algorithms, this research seeks to help leverage the ability of humans to recognize caricatures to improve automatic face recognition methods.

Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

“…Detected facial attributes can be applied directly to authentication. Facial attributes have been applied to enhance face verification, primarily in the case of cross-modal matching, by filtering [19,54] (requiring potential FRF matches to have the correct gender, for example), model switching [18], or aggregation with conventional features [27,17]. [21] defines 65 facial attributes and proposes binary attribute classifiers to predict their presence or absence.…”

“…Because we establish the equivalence between tensor factorisation and gated neural network architecture, our method is scalable to big-data through efficient mini-batch SGD-based learning. In contrast, kernel-based non-linear methods, such as Kernel LDA [34] and multi-kernel SVM [17], are restricted to small data due to their O(N 2 ) computation cost. At runtime, our method only requires a simple feed-forward pass and hence it is also favourable compared to kernel methods.…”

“…For example, the fusion method can be trained with losses known to be effective for face representation learning: identity-supervised softmax, and centre-loss [47]. Alternative methods are either unsupervised [8,27], constrained in the types of supervision they can exploit [3,17], or only stack scores rather than improving a learned representation [48,37]. Therefore, they are relatively ineffective at learning how to combine the two-source information in a task-specific way.…”

Cross-Modal Face Matching: Beyond Viewed Sketches

“…The result outperforms feature-based face-matching techniques, as well as state of the art cross-modal matching techniques that focus on learning a mapping between low-level features without first building an invariant mid-level representation. Moreover, a new dataset combining common forensic [9] and caricature datasets [8] were annotated (≈ 59, 000 annotations in total) to learn and evaluate the proposed cross-modal face representation.…”

Suspect identification based on descriptive facial attributes

“…As previous research has suggested [20], caricature recognition is enabled by targeting the postulated sparse encoding of predominant facial attributes in the human brain. The facial attributes developed in this project were motivated by such findings, and expand on other research related to matching caricatures to photographs [10]. Section 4 discusses the development of these attributes, the use of crowd sourced annotation to label a corpus of data, and provides an analysis of the consistency and discriminability of the choosen attributes.…”