Leveraging mid-level deep representations for predicting face attributes in the wild

Yang, Zhong; Sullivan, Josephine; Li, Haibo

doi:10.1109/icip.2016.7532958

Cited by 40 publications

(42 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Original Pre-cropped FaceTracer [11] 18.88 -PANDA [25] 15.00 -Liu et al [12] 12.70 -Wang et al [63] 12.00 -Zhong et al [64] 10.20 -Rudd et al [ pooling. This verifies that the improvement obtained by our proposed models is due to their content aware pooling/gating mechanisms.…”

Section: Methodsmentioning

confidence: 99%

On Symbiosis of Attribute Prediction and Semantic Segmentation

Kalayeh¹,

Shah²

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Attributes are semantically meaningful characteristics whose applicability widely crosses category boundaries. They are particularly important in describing and recognizing concepts for which no explicit training example is given, e.g., zero-shot learning. Additionally, since attributes are human describable, they can be used for efficient human-computer interaction. In this paper, we propose to employ semantic segmentation to improve person-related attribute prediction. The core idea lies in the fact that many attributes describe local properties. In other words, the probability of an attribute to appear in an image is far from being uniform in the spatial domain. We build our attribute prediction model jointly with a deep semantic segmentation network. This harnesses the localization cues learned by the semantic segmentation to guide the attention of the attribute prediction to the regions where different attributes naturally show up. As a result of this approach, in addition to prediction, we are able to localize the attributes despite merely having access to image-level labels (weak supervision) during training. We first propose semantic segmentation-based pooling and gating, respectively denoted as SSP and SSG. In the former, the estimated segmentation masks are used to pool the final activations of the attribute prediction network, from multiple semantically homogeneous regions. This is in contrast to global average pooling which is agnostic with respect to where in the spatial domain activations occur. In SSG, the same idea is applied to the intermediate layers of the network. Specifically, we create multiple copies of the internal activations. In each copy, only values that fall within a certain semantic region are preserved while outside of that, activations are suppressed. This mechanism allows us to prevent pooling operation from blending activations that are associated with semantically different regions. SSP and SSG, while effective, impose heavy memory utilization since each channel of the activations is pooled/gated with all the semantic segmentation masks. To circumvent this, we propose Symbiotic Augmentation (SA), where we learn only one mask per activation channel. SA allows the model to either pick one, or combine (weighted superposition) multiple semantic maps, in order to generate the proper mask for each channel. SA simultaneously applies the same mechanism to the reverse problem by leveraging output logits of attribute prediction to guide the semantic segmentation task. We evaluate our proposed methods for facial attributes on CelebA and LFWA datasets, while benchmarking WIDER Attribute and Berkeley Attributes of People for whole body attributes. Our proposed methods achieve superior results compared to the previous works. Furthermore, we show that in the reverse problem, semantic face parsing significantly improves when its associated task is jointly learned, through our proposed Symbiotic Augmentation (SA), with facial attribute prediction. We confirm that when few training instances are ...

show abstract

Section: Methodsmentioning

confidence: 99%

On Symbiosis of Attribute Prediction and Semantic Segmentation

Kalayeh¹,

Shah²

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Multiple prediction scores are calculated with the MSE loss to reduce the regression error. Similarly, Zhong et al [135] replace the high-level CNN features in MOON with mid-level features to identify the best representation for each attribute.…”

Section: Holistic Deep Fae Methodsmentioning

confidence: 99%

A Survey of Deep Facial Attribute Analysis

et al. 2020

View full text Add to dashboard Cite

Facial attribute analysis has received considerable attention when deep learning techniques made remarkable breakthroughs in this field over the past few years. Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes. In this paper, we provide a comprehensive survey of deep facial attribute analysis from the perspectives of both estimation and manipulation.First, we summarize a general pipeline that deep facial attribute analysis follows, which comprises two stages: data preprocessing and model construction. Additionally, we introduce the underlying theories of this twostage pipeline for both FAE and FAM. Second, the datasets and performance metrics commonly used in facial attribute analysis are presented. Third, we create a taxonomy of state-of-the-art methods and review deep FAE and FAM algorithms in detail. Furthermore, several additional facial attribute related issues are introduced, as well as relevant real-world applications. Finally, we discuss possible challenges and promising future research directions.

show abstract

“…They have achieved state-of-the-art performance for 40 face attributes prediction tested on CeleA and LFWA, respectively. Using [17], Zhong et al [18] compared different features from different CNN layers and gained a better performance on face attributes prediction using the mid-level CNN feature. More recently, Rudd et al [19] proposed a novel mixed domain adaptive optimization network (MOON) for facial attribute recognition.…”

Section: Related Workmentioning

confidence: 99%

“…Liu et al [17], released the labeled CelebA to the public and they reached 87% accuracy over 40 attributes using LNets+ANet. Zhong et al [18] proposed to leverage the mid-level representations from off-the-shelf architecture to tackle the attribute prediction problem for fasces in the wild. They deployed different deep architecture, but both of them construct SVM as attributes classifier.…”

Section: B Deep Training For Facial Attributes Recognitionmentioning

confidence: 99%

Attributes in Multiple Facial Images

Li¹,

Guo²

2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

View full text Add to dashboard Cite

Facial attribute recognition is conventionally computed from a single image. In practice, each subject may have multiple face images. Taking the eye size as an example, it should not change, but it may have different estimation in multiple images, which would make a negative impact on face recognition. Thus, how to compute these attributes corresponding to each subject rather than each single image is a profound work. To address this question, we deploy deep training for facial attributes prediction, and we explore the inconsistency issue among the attributes computed from each single image. Then, we develop two approaches to address the inconsistency issue. Experimental results show that the proposed methods can handle facial attribute estimation on either multiple still images or video frames, and can correct the incorrectly annotated labels. The experiments are conducted on two large public databases with annotations of facial attributes.

show abstract

Leveraging mid-level deep representations for predicting face attributes in the wild

Cited by 40 publications

References 16 publications

On Symbiosis of Attribute Prediction and Semantic Segmentation

On Symbiosis of Attribute Prediction and Semantic Segmentation

A Survey of Deep Facial Attribute Analysis

Attributes in Multiple Facial Images

Contact Info

Product

Resources

About