Label-Embedding for Image Classification

Abstract

Attributes act as intermediate representations that enable parameter sharing between classes, a must when training data is scarce. We propose to view attribute-based image classification as a label-embedding problem: each class is embedded in the space of attribute vectors. We introduce a function that measures the compatibility between an image and a label embedding. The parameters of this function are learned on a training set of labeled samples to ensure that, given an image, the correct classes rank higher than the incorrect ones. Results on the Animals With Attributes and Caltech-UCSD-Birds datasets show that the proposed framework outperforms the standard Direct Attribute Prediction baseline in a zero-shot learning scenario. Label embedding enjoys a built-in ability to leverage alternative sources of information instead of or in addition to attributes, such as, e.g., class hierarchies or textual descriptions. Moreover, label embedding encompasses the whole range of learning settings from zero-shot learning to regular learning with a large number of labeled examples.

Generalised Zero-Shot Learning with Domain Classification in a Joint Semantic and Visual Space

“…These results are given for the data sets CUB, SUN, AWA1 and AWA2. We compare our approach with 12 leading GZSL methods, which are divided into three groups: semantic (SJE [24], ALE [25], LATEM [26], ES-ZSL [27], SYNC [12], DEVISE [2]), latent space learning (SAE [15], f-CLSWGAN [11], cycle-WGAN [3] and CADA-VAE [4]) and domain classification (CMT [6] and DAZSL [5]). The semantic group contains methods that only use the seen class visual and semantic samples to learn a transformation function from the visual to the semantic space, and classification is based on nearest neighbour classification in that semantic space.…”

DeepKinZero: Zero-Shot Learning for Predicting Kinase-Phosphosite Associations Involving Understudied Kinases

“…The key 70 to making predictions for classes with no training data (referred to as unseen or zero-shot classes) is to have side information which can be used to relate the classes. Based on these relations it becomes possible to transfer the knowledge obtained from classes that have positive training samples (referred to as seen class) [39] to the previously unseen classes.…”

“…In the next section, we elaborate on this approach. Following the work by Akata et al [39], we assume that a vector space representation, called class embedding or kinase embedding, can be constructed for each kinase. Therefore, an m-dimensional "kinase embedding" vector φ(y) ∈ R m can be computed for each kinase y ∈ Y .…”

“…This idea is illustrated in Figure 2. Following the work in structured output prediction [45] and prior work in zero-shot 145 learning [38,39,[46][47][48][49][50], we use a compatibility function F : X × Y → R to model the mapping between the input and output embeddings. In this model, F takes a phosphosite -kinase pair (x i , y j ) as input and returns a scalar value which is proportional to the confidence of associating the site, x i , with kinase y i .…”

“…Zero-shot learning is a machine learning approach that has received significant attention, particularly in the field of computer vision. 70 It handles recognition tasks for classes for which no training examples are available [35][36][37][38][39]. The key to making predictions for classes with no training data (referred to as unseen or zero-shot classes) is to have side information which can be used to relate the classes.…”