Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Hu, Guosheng; Yang, Hua; Yang, Yuan; Zhang, Zhihong; Lü, Zheng; Mukherjee, Sankha; Hospedales, Timothy M.; Robertson, Neil; Yang, Yongxin

doi:10.1109/iccv.2017.404

Cited by 70 publications

(46 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Description 2001 Heckathorn et al [100] First study demonstrating effectiveness of soft biometrics for recognition 2004 Jain et al [93] Bayes rule based model for fingerprint recognition with soft and hard biometrics 2004 Zewail et al [102] Utilized iris color for performing multimodal recognition of fingerprint and iris 2004 Jain et al [103] Utilized gender, height, and ethnicity for fingerprint and face identification 2006 Ailisto et al [104] Incorporated weight and fat percentage for performing fingerprint recognition 2009 Marcialis et al [105] Proposed minority groups to reduce the false rejection rate using soft biometrics 2009 Abreu et al [106] Feature selection using soft biometrics 2010 Moustakas et al [107] User height and stride for supplementing gait recognition 2010 Park and Jain [108] Combined facial marks with an existing face recognition algorithm 2010 Guo et al [101] Analyzed effect of race, gender, height, and weight for cross-age recognition 2011 Scheirer et al [109] Bayesian Attribute Networks for using descriptive attributes for face recognition 2011 Abreu et al [110] Proposed three methods for fusing soft biometric information with primary biometrics 2014 Tome et al [111] Evaluated effect of soft biometrics for recognition from a distance 2015 Tome et al [112] Fusion of continuous and discrete soft biometric traits for face recognition 2017 Mittal et al [113] Utilized soft biometrics for re-ordering the rank list of a face recognition model 2017 Hu et al [114] Tensor-based fusion of face recognition features and face attribute features 2017 Schumann and Stiefelhagen [115] Weighted fusion of attribute prediction and face features for person re-identification 2018 Kazemi et al [116] Attribute centered loss for CNNs: match digital faces with sketch-attribute pairs 2018 Liu et al [117] Attribute guided triplet loss for heterogeneous face matching tion for supplementing gait recognition. A probabilistic framework was used for this purpose.…”

Section: Year Authorsmentioning

confidence: 99%

“…The availability of large-scale datasets with attribute information has further facilitated research in this direction [119]. Hu et al [114] proposed a tensorfusion based framework for combining face recognition and face attribute features, resulting in a Gated Two-stream Neural Network. Schumann and Stiefelhagen [115] proposed learning attribute-complementary features for person re-identification.…”

Section: Year Authorsmentioning

confidence: 99%

See 1 more Smart Citation

A comprehensive overview of biometric fusion

2019

View full text Add to dashboard Cite

The performance of a biometric system that relies on a single biometric modality (e.g., fingerprints only) is often stymied by various factors such as poor data quality or limited scalability. Multibiometric systems utilize the principle of fusion to combine information from multiple sources in order to improve recognition accuracy whilst addressing some of the limitations of singlebiometric systems. The past two decades have witnessed the development of a large number of biometric fusion schemes. This paper presents an overview of biometric fusion with specific focus on three questions: what to fuse, when to fuse, and how to fuse. A comprehensive review of techniques incorporating ancillary information in the biometric recognition pipeline is also presented. In this regard, the following topics are discussed: (i) incorporating data quality in the biometric recognition pipeline; (ii) combining soft biometric attributes with primary biometric identifiers; (iii) utilizing contextual information to improve biometric recognition accuracy; and (iv) performing continuous authentication using ancillary information. In addition, the use of information fusion principles for presentation attack detection and multibiometric cryptosystems is also discussed. Finally, some of the research challenges in biometric fusion are enumerated. The purpose of this article is to provide readers a comprehensive overview of the role of information fusion in biometrics.

show abstract

Section: Year Authorsmentioning

confidence: 99%

Section: Year Authorsmentioning

confidence: 99%

A comprehensive overview of biometric fusion

2019

View full text Add to dashboard Cite

show abstract

“…One related work [33] is to exploit CNN based attribute features for authentication on mobile devices, and the facial attributes are trained by a multi-task, partly based Deep Convolutional Neural Network architecture. Hu et.al [11] systematically study the problem of how to fuse face recognition features and facial attribute features to enhance face recognition performance. They reformulate feature fusion as a gated two-stream neural network, which can be efficiently optimized by neural network learning.…”

Section: Face Recognition With Attributesmentioning

confidence: 99%

Robust RGB-D Face Recognition Using Attribute-Aware Loss

Jiang

Zhang

Deng

2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Existing convolutional neural network (CNN) based face recognition algorithms typically learn a discriminative feature mapping, using a loss function that enforces separation of features from different classes and/or aggregation of features within the same class. However, they may suffer from bias in the training data such as uneven sampling density, because they optimize the adjacency relationship of the learned features without considering the proximity of the underlying faces. Moreover, since they only use facial images for training, the learned feature mapping may not correctly indicate the relationship of other attributes such as gender and ethnicity, which can be important for some face recognition applications. In this paper, we propose a new CNN-based face recognition approach that incorporates such attributes into the training process. Using an attribute-aware loss function that regularizes the feature mapping using attribute proximity, our approach learns more discriminative features that are correlated with the attributes. We train our face recognition model on a large-scale RGB-D data set with over 100K identities captured under real application conditions. By comparing our approach with other methods on a variety of experiments, we demonstrate that depth channel and attribute-aware loss greatly improve the accuracy and robustness of face recognition.

show abstract

“…In the past few years, labeled image datasets have played a critical role in high-level image understanding [Simonyan, 2007;Min, 2016;Zhao, 2018;Zhang, 2017;Xie, 2019;Shu, 2018;Wang, 2015;Hu, 2017;Hua, 2017;Liu, 2018;Huang, 2018;Xu, 2017]. However, the process of constructing manually labeled datasets is both time-consuming and labor-intensive [Deng, 2009].…”

Section: Introductionmentioning

confidence: 99%

“…However, the process of constructing manually labeled datasets is both time-consuming and labor-intensive [Deng, 2009]. To reduce the time and labor cost of manual annotation, learning directly from the web images has attracted more and more attention [Chen, 2013;Yao, 2018;Shen, 2019;Zhang, 2016;Yao, 2019;Liu, 2019;Tang, 2018;Hua, 2017;Xu, 2016;Yang, 2019;Hua, 2016]. Compared to manually-labeled image datasets, web images are a rich and free resource.…”

Section: Introductionmentioning

confidence: 99%

Dynamically Visual Disambiguation of Keyword-based Image Search

Yao

Sun

Shen

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Due to the high cost of manual annotation, learning directly from the web has attracted broad attention. One issue that limits their performance is the problem of visual polysemy. To address this issue, we present an adaptive multi-model framework that resolves polysemy by visual disambiguation. Compared to existing methods, the primary advantage of our approach lies in that our approach can adapt to the dynamic changes in the search results. Our proposed framework consists of two major steps: we first discover and dynamically select the text queries according to the image search results, then we employ the proposed saliency-guided deep multi-instance learning network to remove outliers and learn classification models for visual disambiguation. Extensive experiments demonstrate the superiority of our proposed approach.

show abstract

Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Cited by 70 publications

References 41 publications

A comprehensive overview of biometric fusion

A comprehensive overview of biometric fusion

Robust RGB-D Face Recognition Using Attribute-Aware Loss

Dynamically Visual Disambiguation of Keyword-based Image Search

Contact Info

Product

Resources

About