Inferring Analogous Attributes

Chen, Chao-Yeh; Grauman, Kristen

doi:10.1109/cvpr.2014.33

Cited by 50 publications

(40 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The results demonstrate the advantages of attributes as operators, in terms of the accuracy in recognizing unseen attribute-object compositions. We observe significant improvements over state-of-the-art methods for this task [5,33], with absolute improvements of 3%-12%. Finally, we show that our method is similarly robust whether identifying unseen compositions on their own or in the company of seen compositions-which is of great practical value for recognition in realistic, open world settings.…”

Section: Introductionmentioning

confidence: 85%

Attributes as Operators: Factorizing Unseen Attribute-Object Compositions

Nagarajan

Grauman

2018

Lecture Notes in Computer Science

Self Cite

111

193

View full text Add to dashboard Cite

We present a new approach to modeling visual attributes. Prior work casts attributes in a similar role as objects, learning a latent representation where properties (e.g., sliced) are recognized by classifiers much in the way objects (e.g., apple) are. However, this common approach fails to separate the attributes observed during training from the objects with which they are composed, making it ineffectual when encountering new attribute-object compositions. Instead, we propose to model attributes as operators. Our approach learns a semantic embedding that explicitly factors out attributes from their accompanying objects, and also benefits from novel regularizers expressing attribute operators' effects (e.g., blunt should undo the effects of sharp). Not only does our approach align conceptually with the linguistic role of attributes as modifiers, but it also generalizes to recognize unseen compositions of objects and attributes. We validate our approach on two challenging datasets and demonstrate significant improvements over the state of the art. In addition, we show that not only can our model recognize unseen compositions robustly in an open-world setting, it can also generalize to compositions where objects themselves were unseen during training.

show abstract

Section: Introductionmentioning

confidence: 85%

Attributes as Operators: Factorizing Unseen Attribute-Object Compositions

Nagarajan

Grauman

2018

Lecture Notes in Computer Science

Self Cite

111

193

View full text Add to dashboard Cite

show abstract

“…Accuracy Analogous Attribute [3] 1.4 Red wine [23] 13.1 Attribute as Operator [25] 14.2 VisProd NN [25] 13.9 Label Embedded+ [25] 14.8 TIRG 15.2 Table 3. Comparison to the state-of-the-art on the unseen combination classification task on MIT-States.…”

Section: Methodsmentioning

confidence: 99%

Composing Text and Image for Image Retrieval - an Empirical Odyssey

Jiang

Sun

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

248

304

View full text Add to dashboard Cite

In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image. For example, we may present an image of the Eiffel tower, and ask the system to find images which are visually similar, but are modified in small ways, such as being taken at nighttime instead of during the day. To tackle this task, we learn a similarity metric between a target image x j and a source image x i plus source text t i , i.e., a functionis some representation of the query, such that the similarity is high iff x j is a "positive match" to q i . We propose a new way to combine image and text using f combine , that is designed for the retrieval task. We show this outperforms existing approaches on 3 different datasets, namely Fashion-200k, MIT-States and a new synthetic dataset we create based on CLEVR. We also show that our approach can be used to perform image classification with compositionally novel labels, and we outperform previous methods on MIT-States on this task.

show abstract

“…Wang et al [21] propose a unified probabilistic model to capture the class-dependent and class-independent attribute relationships, which benefit both attribute prediction and object recognition. [5] models high-order relationship between attribute and category to predict category-sensitive attributes and infer unseen category-attribute pairs by using tensor completion based on a sparse set of category-specific attribute classifiers. Fig.…”

Section: Related Workmentioning

confidence: 99%

“…Instead of learning attribute classifier without considering category information, an importance-weighted linear support vector machine is used to predict attribute category-dependently. To train the t th attribute classifier for category j, the violating attribute label constraints for positive and negative samples from category j are given a higher penalty, as suggested in [5].…”

Section: Category-sensitive Attribute Predictionmentioning

confidence: 99%

A Unified Multiplicative Framework for Attribute Learning

Liang

Chang

Shan

et al. 2015

2015 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Attributes are mid-level semantic properties of objects. Recent research has shown that visual attributes can benefit many traditional learning problems in computer vision community. However, attribute learning is still a challenging problem as the attributes may not always be predictable directly from input images and the variation of visual attributes is sometimes large across categories. In this paper, we propose a unified multiplicative framework for attribute learning, which tackles the key problems. Specifically, images and category information are jointly projected into a shared feature space, where the latent factors are disentangled and multiplied for attribute prediction. The resulting attribute classifier is category-specific instead of being shared by all categories. Moreover, our method can leverage auxiliary data to enhance the predictive ability of attribute classifiers, reducing the effort of instance-level attribute annotation to some extent. Experimental results show that our method achieves superior performance on both instance-level and category-level attribute prediction. For zero-shot learning based on attributes, our method significantly improves the state-of-theart performance on AwA dataset and achieves comparable performance on CUB dataset.

show abstract

Inferring Analogous Attributes

Abstract: The appearance of an attribute can vary considerably from class to class (e.g., a "fluffy" dog vs. a "fluffy" towel)

Cited by 50 publications

References 28 publications

Attributes as Operators: Factorizing Unseen Attribute-Object Compositions

Attributes as Operators: Factorizing Unseen Attribute-Object Compositions

Composing Text and Image for Image Retrieval - an Empirical Odyssey

A Unified Multiplicative Framework for Attribute Learning

Contact Info

Product

Resources

About