MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes

Rudd, Ethan M.; Günther, Manuel; Boult, Terrance E.

doi:10.1007/978-3-319-46454-1_2

Cited by 155 publications

(202 citation statements)

References 33 publications

Supporting

Mentioning

194

Contrasting

Order By: Relevance

“…Particularly, we use 40,000 frames with 420 identities for training and the other remaining 20,300 images with 198 identities for testing. Since YTF dataset has not been annotated by facial attributes, we use a well-known attribute prediction model Mixed Objective Optimization Network (MOON) [58] to annotate YTF with the facial attributes and then use it in our model.…”

Section: Resultsmentioning

confidence: 99%

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Taherkhani

Talreja

Valenti

et al. 2020

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

Cross-modal hashing facilitates mapping of heterogeneous multimedia data into a common Hamming space, which can be utilized for fast and flexible retrieval across different modalities. In this paper, we propose a novel cross-modal hashing architecture-deep neural decoder cross-modal hashing (DNDCMH), which uses a binary vector specifying the presence of certain facial attributes as an input query to retrieve relevant face images from a database. The DNDCMH network consists of two separate components: an attribute-based deep cross-modal hashing (ADCMH) module, which uses a margin (m)-based loss function to efficiently learn compact binary codes to preserve similarity between modalities in the Hamming space, and a neural error correcting decoder (NECD), which is an error correcting decoder implemented with a neural network. The goal of NECD network in DNDCMH is to error correct the hash codes generated by ADCMH to improve the retrieval efficiency. The NECD network is trained such that it has an error correcting capability greater than or equal to the margin (m) of the margin-based loss function. This results in NECD can correct the corrupted hash codes generated by ADCMH up to the Hamming distance of m. We have evaluated and compared DNDCMH with state-of-the-art cross-modal hashing methods on standard datasets to demonstrate the superiority of our method.! DCMH [22], the inter-modal triplet embedding loss encourages the heterogeneous correlation across different modalities, and the intra-modal triplet loss encodes the discriminative power of the hash codes. Moreover, a regularization loss is used to apply adjacency consistency to ensure that the hash codes can keep the original similarities in Hamming space. However, in margin-based loss functions, some of the instances of different modalities of the same subject may not be close enough in Hamming space to guarantee all the correct retrievals. Therefore, it is important to bring the different modalities of the same subject closer to each other in Hamming space to improve the retrieval efficiency.In this work, we observe that in addition to the regular DCMH techniques [13], [24], [25], which exploit entropy maximization and quantization losses in the objective function of the DCMH, an error-correcting code (ECC) decoder can be used as an additional component to compensate for the heterogeneity gap and reduce the Hamming distance between the different modalities of the same subject in order to improve the cross-modal retrieval efficiency. We presume that the hash code generated by DCMH is a binary vector that is within a certain distance from a codeword of an ECC. When the hash code generated by DCMH is passed through an ECC decoder, the closest codeword to this hash code is found, which can be used as a final hash code for the retrieval process. In this process, the attribute hash code and image hash code of the same subject are forced to map to the same codeword, thereby reducing the distance of the corresponding hash codes. This brings more relevant facial images ...

show abstract

Section: Resultsmentioning

confidence: 99%

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Taherkhani

Talreja

Valenti

et al. 2020

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

show abstract

“…An alternative to purpose designing CNNs for FAR is to fine-tune networks intended for object recognition [56,57]. From a representation learning perspective, the features supporting different attribute detections may be shared, leading some studies to investigate multi-task learning facial attributes [55,30]. Since different facial attributes have different prevalence, the multi-label/multi-task learning suffers from label-imbalance, which [30] addresses using a mixed objective optimization network (MOON).…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we introduce the CNN architectures used for face recognition (LeanFace) designed by ourselves and facial attribute recognition (AttNet) introduced by [50,30]. LeanFace.…”

Section: Integration With Cnns: Architecturementioning

confidence: 99%

See 1 more Smart Citation

Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Hu¹,

Yang²,

Yang³

et al. 2017

2017 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

“…Therefore, the intrinsic relationships between these tasks are not fully and effectively exploited. Moreover, some multi-label learning based FAC methods (such as [19], [20]) are developed to simultaneously predict facial attributes by using a single CNN. These methods treat the diverse arXiv:2002.03683v1 [cs.CV] 10 Feb 2020 attributes equally (using the same network architecture for all attributes), ignoring the different learning complexities of these attributes (for example, learning to predict the "Wearing-Eyeglasses" attribute may be much easier than identifying the "Pointy Nose" attribute, as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

Deep Multi-Task Multi-Label CNN for Effective Facial Attribute Classification

Mao

Yan

Xue

et al. 2022

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Facial Attribute Classification (FAC) has attracted increasing attention in computer vision and pattern recognition. However, state-of-the-art FAC methods perform face detection/alignment and FAC independently. The inherent dependencies between these tasks are not fully exploited. In addition, most methods predict all facial attributes using the same CNN network architecture, which ignores the different learning complexities of facial attributes. To address the above problems, we propose a novel deep multi-task multi-label CNN, termed DMM-CNN, for effective FAC. Specifically, DMM-CNN jointly optimizes two closely-related tasks (i.e., facial landmark detection and FAC) to improve the performance of FAC by taking advantage of multi-task learning. To deal with the diverse learning complexities of facial attributes, we divide the attributes into two groups: objective attributes and subjective attributes. Two different network architectures are respectively designed to extract features for two groups of attributes, and a novel dynamic weighting scheme is proposed to automatically assign the loss weight to each facial attribute during training. Furthermore, an adaptive thresholding strategy is developed to effectively alleviate the problem of class imbalance for multi-label learning. Experimental results on the challenging CelebA and LFWA datasets show the superiority of the proposed DMM-CNN method compared with several state-of-the-art FAC methods.

show abstract

MOON: A Mixed Objective Optimization Network for the Recognition of Facial Attributes

Cited by 155 publications

References 33 publications

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Deep Multi-Task Multi-Label CNN for Effective Facial Attribute Classification

Contact Info

Product

Resources

About