Deep Multi-Task Multi-Label CNN for Effective Facial Attribute Classification

Abstract

Facial Attribute Classification (FAC) has attracted increasing attention in computer vision and pattern recognition. However, state-of-the-art FAC methods perform face detection/alignment and FAC independently. The inherent dependencies between these tasks are not fully exploited. In addition, most methods predict all facial attributes using the same CNN network architecture, which ignores the different learning complexities of facial attributes. To address the above problems, we propose a novel deep multi-task multi-label CNN, termed DMM-CNN, for effective FAC. Specifically, DMM-CNN jointly optimizes two closely-related tasks (i.e., facial landmark detection and FAC) to improve the performance of FAC by taking advantage of multi-task learning. To deal with the diverse learning complexities of facial attributes, we divide the attributes into two groups: objective attributes and subjective attributes. Two different network architectures are respectively designed to extract features for two groups of attributes, and a novel dynamic weighting scheme is proposed to automatically assign the loss weight to each facial attribute during training. Furthermore, an adaptive thresholding strategy is developed to effectively alleviate the problem of class imbalance for multi-label learning. Experimental results on the challenging CelebA and LFWA datasets show the superiority of the proposed DMM-CNN method compared with several state-of-the-art FAC methods.

Learning Spatial-Semantic Relationship for Facial Attribute Recognition with Limited Labeled Data

“…In this section, we compare the proposed SSPL method with ten state-of-the-art methods, including five supervised FAR methods [22,27,20,2,14], three self-supervised learning methods [3,24,10], and two semi-supervised learning methods [28,23], on the CelebA and LFWA 2.…”

“…Sharma and Foroosh [27] leverage deep separable convolutions and pointwise convolution to design a lightweight CNN for FAR, which significantly reduces the model parameters and improves the computational efficiency. Mao et al [22] perform FAR based on a Deep Multi-task and Multi-label Convolutional Neural Network (DMM-CNN). He et al [14] propose to use synthesized abstraction images to improve the FAR performance.…”

Augmented binary multi-labeled CNN for practical facial attribute classification

“…Mao et al, propose a new algorithm to deal with the face attribute extraction from facial images, the algorithm called deep multi-task multi-label CNN (DMM-CNN), by dividing the facial attributes in two categories objective and subjective they manage to run two different network architectures taking advantage of multitask learning, and adopting dynamic weighting scheme to resolve the problem of diverse learning complexities [8]. In the same field Ehrlich and Shields propose there multitask learning facial attributes approach [15], while Mandel, Pascanu propose a method based on shared features between these attributes, by using multi-task restricted boltzmann machine (MT-RBM) [16], they were able to learning a joint feature representation from facial landmark points for all attributes, following by approach subsist of a bottomup/top-down pass for learning the shared representation of multitask models, and bottom-up pass for prediction of tasks.…”

“…The main property of FAC [5] is to predict multiple face features, state and emotion [6] on the given image or face portrait. Various algorithms have reached an excellent result on multiple levels for FAC, either apply directly CNN [7] models to extract face features, or using methods for improving the learning by distributing the attributes into two categories: objective attributes like wearing a hat, eyeglasses, bangs and subjective ones like smiling, big lips [8]. Some methods focus on grouping some attributes on the basis of their intercorrelations [9], while others target detecting face landmark localization [10] to reduce the noise.…”

Let Them Choose What They Want: A Multi-Task CNN Architecture Leveraging Mid-Level Deep Representations for Face Attribute Classification

“…To find out which level deep representation is the best for FAC when fused with the high-level one, we make 3 sets of experiments: In a word, the proposed method shows superiority in FAC. [15] 91.26 320 MCNN-AUX [15] 91.29 -MCFA [16] 91.23 -DMM-CNN [17] 91.70 -PS-MCNN [18] 92.22 16 Our Method 92.35 16…”