Zero-Shot Visual Recognition via Bidirectional Latent Embedding

Abstract

Zero-shot learning for visual recognition, e.g., object and action recognition, has recently attracted a lot of attention. However, it still remains challenging in bridging the semantic gap between visual features and their underlying semantics and transferring knowledge to semantic categories unseen during learning. Unlike most of the existing zero-shot visual recognition methods, we propose a stagewise bidirectional latent embedding framework to two subsequent learning stages for zero-shot visual recognition. In the bottom-up stage, a latent embedding space is first created by exploring the topological and labeling information underlying training data of known classes via a proper supervised subspace learning algorithm and the latent embedding of training data are used to form landmarks that guide embedding semantics underlying unseen classes into this learned latent space. In the top-down stage, semantic representations of unseen-class labels in a given label vocabulary are then embedded to the same latent space to preserve the semantic relatedness between all different classes via our proposed semi-supervised Sammon mapping with the guidance of landmarks. Thus, the resultant latent embedding space allows for predicting the label of a test instance with a simple nearest-neighbor rule. To evaluate the effectiveness of the proposed framework, we have conducted extensive experiments on four benchmark datasets in object and action recognition, i.e., AwA, CUB-200-2011, UCF101 and HMDB51. The experimental results under comparative studies demonstrate that our proposed approach yields the state-of-the-art performance under inductive and transductive settings.

An approach for adaptive automatic threat recognition within 3D computed tomography images for baggage security screening

“…In our current approach, we have not explicitly handled the CT image noises such as metal artefacts which have been reported affecting the segmentation [20]. To enhance the adaptability to new materials, the techniques of zero-shot learning [37] and few-shot learning [36] can also be employed in future work.…”

Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition

“…To facilitate the separability of data projected into the subspace, we follow [17] and apply the centralization (i.e. mean subtraction) and l2 normalization to all the projections:…”

“…Note that the class meansz y are calculated using only labelled source data for unsupervised domain adaptation and labelled target data are also used for zero-shot learning problem. Following [17], we apply l2 normalization toz y before using them in Eq.(9). To this point, it is straightforward to apply the proposed approach to the zero-shot learning condition and the algorithm is summarized in Algorithm 1.…”

“…To adapt them to our problem, we calculate class-level source domain representations by averaging the source domain samples belonging to the same class. After this adaptation, we apply two zero-shot learning algorithms, i.e., bidirectional latent embedding learning (BiDiLEL) [17] and manifold regularized ridge regression (MR) [38] to our problem for the comparison. We also replace the SLPP with LDA in our framework to validate the importance of preserving data structure when learning subspace of favourable properties.…”

“…Our approach aims to learn a subspace in which the domain and target data can be aligned and wellseparated. To this end, a supervised locality preserving projection (LPP) [17], [18] is employed as an enabling technique for subspace learning. For unsupervised domain adaptation, we propose a confidence-aware pseudo label selection scheme to gradually align the domains in an iterative learning strategy.…”