How are attributes expressed in face DCNNs?

Dhar, Prithviraj; Bansal, Ankan; Castillo, Carlos D.; Gleason, Joshua; Phillips, P. Jonathon; Chellappa, Rama

doi:10.1109/fg47880.2020.00009

Cited by 24 publications

(22 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Open questions remain about how to optimize models of the visual processing of faces. In the present study, and in others ( Colón et al., 2021 ; Dhar et al., 2020 ; Hill et al., 2019 ; Parde et al., 2017 , 2019 ), face representations were built from deep learning algorithms trained explicitly to separate identities. In other work (Yildirim et al., 2020) , networks have been trained to simultaneously decode an array of face parameters from images (i.e., shape and texture coefficients from a three-dimensional computer graphics model of faces, along with illumination direction and pose angle).…”

Section: Discussionmentioning

confidence: 96%

Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition

et al. 2021

Self Cite

View full text Add to dashboard Cite

Single-unit responses and population codes differ in the “read-out” information they provide about high-level visual representations. Diverging local and global read-outs can be difficult to reconcile with in vivo methods. To bridge this gap, we studied the relationship between single-unit and ensemble codes for identity, gender, and viewpoint, using a deep convolutional neural network (DCNN) trained for face recognition. Analogous to the primate visual system, DCNNs develop representations that generalize over image variation, while retaining subject (e.g., gender) and image (e.g., viewpoint) information. At the unit level, we measured the number of single units needed to predict attributes (identity, gender, viewpoint) and the predictive value of individual units for each attribute. Identification was remarkably accurate using random samples of only 3% of the network's output units, and all units had substantial identity-predicting power. Cross-unit responses were minimally correlated, indicating that single units code non-redundant identity cues. Gender and viewpoint classification required large-scale pooling of units—individual units had weak predictive power. At the ensemble level, principal component analysis of face representations showed that identity, gender, and viewpoint separated into high-dimensional subspaces, ordered by explained variance. Unit-based directions in the representational space were compared with the directions associated with the attributes. Identity, gender, and viewpoint contributed to all individual unit responses, undercutting a neural tuning analogy. Instead, single-unit responses carry superimposed, distributed codes for face identity, gender, and viewpoint. This undermines confidence in the interpretation of neural representations from unit response profiles for both DCNNs and, by analogy, high-level vision.

show abstract

Section: Discussionmentioning

confidence: 96%

Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…This process becomes even more challenging when derived data representations are considered, i.e., templates. As shown by recent work [59], [60], contemporary biometric templates encode a variety of information that is entangled in the features and, therefore, difficult to remove or hide in a selective manner.…”

Section: Biometric Privacy Enhancementmentioning

confidence: 99%

Privacy–Enhancing Face Biometrics: A Comprehensive Survey

Meden

Rot

Terhörst

et al. 2021

IEEE Trans.Inform.Forensic Secur.

108

View full text Add to dashboard Cite

Biometric recognition technology has made significant advances over the last decade and is now used across a number of services and applications. However, this widespread deployment has also resulted in privacy concerns and evolving societal expectations about the appropriate use of the technology. For example, the ability to automatically extract age, gender, race, and health cues from biometric data has heightened concerns about privacy leakage. Face recognition technology, in particular, has been in the spotlight, and is now seen by many as posing a considerable risk to personal privacy. In response to these and similar concerns, researchers have intensified efforts towards developing techniques and computational models capable of ensuring privacy to individuals, while still facilitating the utility of face recognition technology in several application scenarios. These efforts have resulted in a multitude of privacy-enhancing techniques that aim at addressing privacy risks originating from biometric systems and providing technological solutions for legislative requirements set forth in privacy laws and regulations, such as GDPR. The goal of this overview paper is to provide a comprehensive introduction into privacy-related research in the area of biometrics and review existing work on Biometric Privacy-Enhancing Techniques (B-PETs) applied to face biometrics. To make this work useful for as wide of an audience as possible, several key topics are covered as well, including evaluation strategies used with B-PETs, existing datasets, relevant standards, and regulations and critical open issues that will have to be addressed in the future.

show abstract

“…Recently, Dhar et al [120] found that age-related information is highly coupled with identity-salient features in the latent space of a well-trained face recognition model [121]. The entanglement of these attributes has been exploited by Deb et al [113], who trained an autoencoder that operates directly within the latent space of a pre-trained face recognition model, such as CosNet [114].…”

Section: Child Vs Adult Face Ageingmentioning

confidence: 99%

Deep Face Age Progression: A Survey

2021

View full text Add to dashboard Cite

Face Age Progression (FAP) refers to synthesizing face images while simulating ageing effects, thus enabling predicting the future appearance of an individual. The generation of age-progressed face images brings benefits for various applications, ranging from face recognition systems to forensic investigations and digital entertainment. In particular, the recent success achieved with deep generative networks significantly leveraged the quality of age-synthesized face images in terms of visual fidelity, ageing accuracy and identity preservation. However, the high number of contributions in recent years requires systematically structuring new findings and ideas to identify a common taxonomy, accelerate future research and reduce redundancy. Therefore, we present a comparative analysis of recent deep learning based face age progression methods for both adult and child-based face ageing, broken down into three high-level concepts: translation-based, condition-based, and sequence-based FAP. Further, we offer a comprehensive summary of the most common performance evaluation techniques, cross-age datasets, and open challenges to steer future research in the right direction.

show abstract

How are attributes expressed in face DCNNs?

Cited by 24 publications

References 26 publications

Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition

Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition

Privacy–Enhancing Face Biometrics: A Comprehensive Survey

Deep Face Age Progression: A Survey

Contact Info

Product

Resources

About