Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach

Abstract

Abstract-Face attribute estimation has many potential applications in video surveillance, face retrieval, and social media. While a number of methods have been proposed for face attribute estimation, most of them did not explicitly consider the attribute correlation and heterogeneity (e.g., ordinal vs. nominal and holistic vs. local) during feature representation learning. In this paper, we present a Deep Multi-Task Learning (DMTL) approach to jointly estimate multiple heterogeneous attributes from a single face image. In DMTL, we tackle attribute correlation and heterogeneity with convolutional neural networks (CNNs) consisting of shared feature learning for all the attributes, and category-specific feature learning for heterogeneous attributes. We also introduce an unconstrained face database (LFW+), an extension of public-domain LFW, with heterogeneous demographic attributes (age, gender, and race) obtained via crowdsourcing. Experimental results on benchmarks with multiple face attributes (MORPH II, LFW+, CelebA, LFWA, and FotW) show that the proposed approach has superior performance compared to state of the art. Finally, evaluations on a public-domain face database (LAP) with a single attribute show that the proposed approach has excellent generalization ability.

Transferability and Hardness of Supervised Classification Tasks

“…Finally, we note that our goal is not to develop a state of the art facial attribute classification scheme. Nevertheless, results obtained by training an lSVM on embeddings transferred from a face recognition network are only 2.4% lower than the best scores reported by DMTL 2018 [22] (last column of Table 1). The effort involved in developing a state of the art face recognition network can be substantial.…”

C3AE: Exploring the Limits of Compact Model for Age Estimation

“…Following [24], Chen et al [3] utilized ranking-CNN for age estimation, in which there were a series of basic binary CNNs, aggregating to the final estimation. Han et al [9] used multiple attributes for multi-task learning. Gao et al [6] used KL divergence to measure the similarity between the estimated and groundtruth distributions for age.…”

Registration-free Face-SSD: Single shot analysis of smiles, facial attributes, and affect in the wild

“…Registration-based face analysis. Despite significant advances in deep learning, automatic face analysis tasks, such as smile detection (a comparative review is provided in Table 3), attribute prediction [12,11,43] and valence-arousal estimation [32], still face major challenges caused by occlusions and variances of head pose, scale, and illumination. These challenges are the main reason why every state-of-the-art approach to face analysis requires a pre-normalisation step involving face detection and registration (rotation, scaling, and 2D/3D transformation).…”

“…Several recent approaches address the problem of facial attributes prediction [11,43]. Some propose to use successful face-specific feature representations [64], modelling class distributions [8] and balancing attributes [37], indirect guiding the categorisation of similar features [52,43], or direct grouping the relevant attributes [12,11]. The best performances (more than 90% accuracy) are obtained by specifically designing a model structure that utilises the relations between relevant attributes [12,11].…”