Decorrelating Semantic Visual Attributes by Resisting the Urge to Share

Jayaraman, Dinesh; Sha, Fei; Grauman, Kristen

doi:10.1109/cvpr.2014.211

Cited by 133 publications

(167 citation statements)

References 14 publications

Supporting

Mentioning

165

Contrasting

Unclassified

Order By: Relevance

“…[23] proposed a model DKRL, combining the existing model TransE (originally used for KG completion) [3] and CNN (or BOW), for KGE in zero-shot scenario. In computer vision, [24], [25] train a recognition model for zero-shot object recognition by specifying the category's attributes. [26] proposes a label-embedding model for attribute-based zero-shot classification.…”

Section: Related Workmentioning

confidence: 99%

Zero-Shot Embedding for Unseen Entities in Knowledge Graph

Gao

Gallinari

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYKnowledge graph (KG) embedding aims at learning the latent semantic representations for entities and relations. However, most existing approaches can only be applied to KG completion, so cannot identify relations including unseen entities (or Out-of-KG entities). In this paper, motivated by the zero-shot learning, we propose a novel model, namely JointE, jointly learning KG and entity descriptions embedding, to extend KG by adding new relations with Out-of-KG entities. The JointE model is evaluated on entity prediction for zero-shot embedding. Empirical comparisons on benchmark datasets show that the proposed JointE model outperforms state-of-the-art approaches. The source code of JointE is available at https://github.com/yzur/JointE.

show abstract

Section: Related Workmentioning

confidence: 99%

Zero-Shot Embedding for Unseen Entities in Knowledge Graph

Gao

Gallinari

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…During the test, a prediction can be made by Maximum-a-Posteriori criteria over all of the outputs of the binary classifiers. The main drawback of such framework is the correlation problem that reported in [10]. Besides, the human-defined attribute list can be unrealistic and noisy and need to be selected [9,7,16,18].…”

Section: Related Workmentioning

confidence: 99%

Towards Fine-Grained Open Zero-Shot Learning: Inferring Unseen Visual Features from Attributes

Long

Liu

Shao³

2017

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Zero-shot Learning (ZSL) can leverage attributes to recognise unseen instances. However, the training data is limited and cannot adequately discriminate fine-grained classes with similar attributes. In this paper, we propose a complementary procedure that inversely makes use of attributes to infer discriminative visual features for unseen classes. In this way, ZSL is fully converted into conventional supervised classification, where robust classifiers can be employed to address the fine-grained problem. To infer high-quality unseen data, we propose a novel algorithm named Orthogonal Semantic-Visual Embedding (OSVE) that can discover the tiny visual differences between different instances under the same attribute by an orthogonal embedding space. On two fine-grained benchmarks, CUB and SUN, our method remarkably improves the state-of-the-art results under standard ZSL settings. We further challenge the Open ZSL problem where the number of seen classes is significantly smaller than that of unseen classes. Substantial experiments manifest that the inferred visual features can be successfully fed to SVM which can effectively discriminate unseen classes from fine-grained open candidates.

show abstract

“…To further prove that our part-aware method somehow decorrelates the attributes, we evaluate against the state-of-the-art attribute decorrelation method introduced in [18], where they use semantic groups to encourage in-group feature sharing and between-group competition for features through a lasso multi-task learning framework. We compare with two variants of their method (i) similar to [18], when holistic image-wide features divided into 6 regular grids are used (Weakly-Supervised (WS)-Decor), and (ii) when ground-truth part annotations are supplied to extract part-level features (StronglySupervised (SS)-Decor). We also compare performance of strongly-supervised DPM against the original weaklysupervised DPM [11] which works without strong part annotations at training (Weakly-Supervised (WS)-DPM).…”

Section: Baselines (Attribute Detection)mentioning

confidence: 99%

“…Moreover, such attributes may provide a route to bridge the sketch/photo modality gap, as they are domain invariant if reliably detected (e.g., a high-heel shoe is 'high-heel' regardless if depicted in a photo or sketch). However, they suffer from being hard to predict due to spurious correlations [18]. In this paper we bring together attribute and part-centric modeling to decorrelate and better predict attributes, as well as provide two complementary views of the data to enhance matching.…”

Section: Introductionmentioning

confidence: 99%

Fine-grained sketch-based image retrieval: The role of part-aware attributes

Pang

Song

et al. 2016

2016 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

We study the problem of fine-grained sketch-based image retrieval. By performing instance-level (rather than category-level) retrieval, it embodies a timely and practical application, particularly with the ubiquitous availability of touchscreens. Three factors contribute to the challenging nature of the problem: (i) free-hand sketches are inherently abstract and iconic, making visual comparisons with photos more difficult, (ii) sketches and photos are in two different visual domains, i.e. black and white lines vs. color pixels, and (iii) fine-grained distinctions are especially challenging when executed across domain and abstraction-level. To address this, we propose to detect visual attributes at part-level, in order to build a new representation that not only captures fine-grained characteristics but also traverses across visual domains. More specifically, (i) we propose a dataset with 304 photos and 912 sketches, where each sketch and photo is annotated with its semantic parts and associated part-level attributes, and with the help of this dataset, we investigate (ii) how strongly-supervised deformable part-based models can be learned that subsequently enable automatic detection of part-level attributes, and (iii) a novel matching framework that synergistically integrates low-level features, mid-level geometric structure and high-level semantic attributes to boost retrieval performance. Extensive experiments conducted on our new dataset demonstrate value of the proposed method.

show abstract

Decorrelating Semantic Visual Attributes by Resisting the Urge to Share

Cited by 133 publications

References 14 publications

Zero-Shot Embedding for Unseen Entities in Knowledge Graph

Zero-Shot Embedding for Unseen Entities in Knowledge Graph

Towards Fine-Grained Open Zero-Shot Learning: Inferring Unseen Visual Features from Attributes

Fine-grained sketch-based image retrieval: The role of part-aware attributes

Contact Info

Product

Resources

About