Synthesized Classifiers for Zero-Shot Learning

IEEE Trans. Pattern Anal. Mach. Intell.

Liu

Shen

et al. 2018

Abstract-Sufficient training examples are the fundamental requirement for most of the learning tasks. However, collecting welllabelled training examples is costly. Inspired by Zero-shot Learning (ZSL) that can make use of visual attributes or natural language semantics as an intermediate level clue to associate low-level features with high-level classes, in a novel extension of this idea, we aim to synthesise training data for novel classes using only semantic attributes. Despite the simplicity of this idea, there are several challenges. Firstly, how to prevent the synthesised data from over-fitting to training classes? Secondly, how to guarantee the synthesised data is discriminative for ZSL tasks? Thirdly, we observe that only a few dimensions of the learnt features gain high variances whereas most of the remaining dimensions are not informative. Thus, the question is how to make the concentrated information diffuse to most of the dimensions of synthesised data. To address the above issues, we propose a novel embedding algorithm named Unseen Visual Data Synthesis (UVDS) that projects semantic features to the high-dimensional visual feature space. Two main techniques are introduced in our proposed algorithm. (1) We introduce a latent embedding space which aims to reconcile the structural difference between the visual and semantic spaces, meanwhile preserve the local structure. (2) We propose a novel Diffusion Regularisation (DR) that explicitly forces the variances to diffuse over most dimensions of the synthesised data. By an orthogonal rotation (more precisely, an orthogonal transformation), DR can remove the redundant correlated attributes and further alleviate the over-fitting problem. On four benchmark datasets, we demonstrate the benefit of using synthesised unseen data for zero-shot learning. Extensive experimental results suggest that our proposed approach significantly outperforms the state-of-the-art methods.

Section: Related Workmentioning

confidence: 99%

Zero-Shot Learning Using Synthesised Unseen Visual Data with Diffusion Regularisation

IEEE Trans. Pattern Anal. Mach. Intell.

Liu

Shen

et al. 2018

“…Many recent approaches adopt such an embedding manner and achieve promising results [13,4,33,15,7,19,39,8,23]. Besides, similarity-based frameworks also adopt the embedding approach [24,40,41,34,8,25]. But the semantic space aims to associate unseen to seen classes.…”

Section: Related Workmentioning

confidence: 99%

“…the trained model on seen classes is also effective on unseen classes; 2) visual-related, the gap between the semantic and visual spaces should be small enough to train a stable model. According to these requirements, learning visual attributes has gain most popularity [21,29,38,27,14,16,8]. However, attribute annotations are very expensive, especially for image-level tasks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Describing Unseen Classes by Exemplars: Zero-Shot Learning Using Grouped Simile Ensemble

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

Shao

2017

Learning visual attributes is an effective approach for zero-shot recognition. However, existing methods are restricted to learning explicitly nameable attributes and cannot tell which attributes are more important to the recognition task. In this paper, we propose a unified framework named Grouped Simile Ensemble (GSE). We claim our contributions as follows. 1) We propose to substitute explicit attribute annotation by similes, which are more natural expressions that can describe complex unseen classes. Similes do not involve extra concepts of attributes, i.e. only exemplars of seen classes are needed. We provide an efficient scenario to annotate similes for two benchmark datasets, AwA and aPY. 2) We propose a graph-cut-based class clustering algorithm to effectively discover implicit attributes from the similes. 3) Our GSE can automatically find the most effective simile groups to make the prediction. On both datasets, extensive experimental results manifest that our approach can significantly improve the performance over the state-of-the-art methods.

“…[35,4] proposes bilinear joint embeddings to mitigate the distribution difference between visual and semantic spaces. In [5], classifiers of unseen classes are directly estimated by aligning the manifolds of seen classes.…”

Section: Related Workmentioning

confidence: 99%

Towards Fine-Grained Open Zero-Shot Learning: Inferring Unseen Visual Features from Attributes

2017 IEEE Winter Conference on Applications of Computer Vision (WACV)

Liu

Shao³

2017

Zero-shot Learning (ZSL) can leverage attributes to recognise unseen instances. However, the training data is limited and cannot adequately discriminate fine-grained classes with similar attributes. In this paper, we propose a complementary procedure that inversely makes use of attributes to infer discriminative visual features for unseen classes. In this way, ZSL is fully converted into conventional supervised classification, where robust classifiers can be employed to address the fine-grained problem. To infer high-quality unseen data, we propose a novel algorithm named Orthogonal Semantic-Visual Embedding (OSVE) that can discover the tiny visual differences between different instances under the same attribute by an orthogonal embedding space. On two fine-grained benchmarks, CUB and SUN, our method remarkably improves the state-of-the-art results under standard ZSL settings. We further challenge the Open ZSL problem where the number of seen classes is significantly smaller than that of unseen classes. Substantial experiments manifest that the inferred visual features can be successfully fed to SVM which can effectively discriminate unseen classes from fine-grained open candidates.