Efficient Relative Attribute Learning Using Graph Neural Networks

Abstract

A sizable body of work on relative attributes provides evidence that relating pairs of images along a continuum of strength pertaining to a visual attribute yields improvements in a variety of vision tasks. In this paper, we show how emerging ideas in graph neural networks can yield a solution to various problems that broadly fall under relative attribute learning. Our main idea is the observation that relative attribute learning naturally benefits from exploiting the graph of dependencies among the different relative attributes of images, especially when only partial ordering is provided at training time. We use message passing to perform end to end learning of the image representations, their relationships as well as the interplay between different attributes. Our experiments show that this simple framework is effective in achieving competitive accuracy with specialized methods for both relative attribute learning and binary attribute prediction, while relaxing the requirements on the training data and/or the number of parameters, or both.

Estimation of Semantic Impressions from Portraits

“…Relative attributes are used in various tasks that attempt to infer the strength of attributes: skill determination [17], prediction of a person's beauty [18], [19], prediction of an image's aesthetics [20], [21], and prediction of an image's attributes [22]- [25]. For handling relative attributes, learning-to-rank algorithms are generally used and work effectively.…”

“…Yu and Grauman [28], [29] attempted to increase the data by generating synthetic images based on attributes and to learn the discrimination using local similarity the same as the method [27]. In another approach, Meng et al [22] performed multi-task learning using a graph-based neural network. Miyata et al [14] developed a ranking network to learn the difference of the impression between a pair of deformed portraits.…”

A Survey of Deep Facial Attribute Analysis

“…To model the pair-wise relationships between images for multiple attributes, Meng et al [77] construct a graph model, where each node represents an image and edges indicate the relationships between images and attributes, as well as between images and images. The overall framework consists of two components: the CNN for extracting primary features of the node images, and the graph neural network (GNN) for learning the features of edges and following updates.…”

Hybrid Graph Neural Networks for Crowd Counting

“…Gilmer et al (Gilmer et al 2017) used message passing neural network to generalize the GNN. Recently, GNN has been successfully applied in attributes recognition (Meng et al 2018), humanobject interactions (Qi et al 2018a), action recognition (Si et al 2018), etc. Our HyGnn shares similar ideas with A novel HyGnn is used to distill multi-scale and cross-domain information, so as to learn better representations.…”