Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective

Abstract

Zero-shot learning (ZSL) aims to recognize instances of unseen classes solely based on the semantic descriptions of the classes. Existing algorithms usually formulate it as a semantic-visual correspondence problem, by learning mappings from one feature space to the other. Despite being reasonable, previous approaches essentially discard the highly precious discriminative power of visual features in an implicit way, and thus produce undesirable results. We instead reformulate ZSL as a conditioned visual classification problem, i.e., classifying visual features based on the classifiers learned from the semantic descriptions. With this reformulation, we develop algorithms targeting various ZSL settings: For the conventional setting, we propose to train a deep neural network that directly generates visual feature classifiers from the semantic attributes with an episode-based training scheme; For the generalized setting, we concatenate the learned highly discriminative classifiers for seen classes and the generated classifiers for unseen classes to classify visual features of all classes; For the transductive setting, we exploit unlabeled data to effectively calibrate the classifier generator using a novel learning-without-forgetting self-training mechanism and guide the process by a robust generalized cross-entropy loss. Extensive experiments show that our proposed algorithms significantly outperform state-of-the-art methods by large margins on most benchmark datasets in all the ZSL settings. Our code is available at https://github. com/kailigo/cvcZSL

Multimodal Style Transfer via Graph Cuts

“…Originating from non-realistic rendering [18], image style transfer is closely related to texture synthesis [5,7,6]. Gatys et al [8] were the first to formulate style transfer as the matching of multi-level deep features extracted from a pre-trained deep neural network, which has been widely used in various tasks [20,21,22]. Lots of improvements have been proposed based on the works of Gatys et al [8].…”

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

“…In other words, it addresses multi-class learning problems when some classes do not have sufficient training data. However, during the learning process, additional visual and semantic features such as word embeddings [132], visual attributes [133], or descriptions [134] can be assigned to both seen and unseen classes. In the context of multimodality, a multimodal mapping scheme typically combines visual and semantic attributes using only data related to the seen classes.…”

A Single Historical Painting Super-Resolution via a Reference-Based Zero-Shot Network

“…The application of zero-shot in super-resolution, i.e, the ZSSR prescription [23], is among the most widely used models in superresolution and has gained increasing interest recently. In addition, the majority of zero-shot methods that have provided a large number of excellent results in recent years are mainly based on segmentation [39], emotion recognition [40], object detection [41], image retrieval [42][43][44][45], image classification [46][47][48] and intelligent learning in machines or robots [49]. In the ZSSR formalism, LR images are downsampled to generate many lower-resolution images (I = I 0 , I 1 , I 2 , ..., I n ), which serve as the HR supervision information called "HR fathers, " then, each HR father is downscaled by the required scale factor s to obtain the corresponding "LR sons.…”