Attribute-Guided Network for Cross-Modal Zero-Shot Hashing

Ji, Zhong; Sun, Yuxin; Yu, Yunlong; Pang, Yanwei; Han, Jungong

doi:10.1109/tnnls.2019.2904991

Cited by 75 publications

(31 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The semantic and visual information are regarded as different modalities, and autoencoders are constructed to map them into a common latent encoding space. Reference [26] aligned different modalities into an attribute space and constructed hash codes of images and texts respectively to realize bidirectional retrieval. Furthermore, in [27], it was considered that the given class-related attributes were too subjective to distinguish target classes in ZSL, so the paper elaborated an optimization problem to learn more potential attributes.…”

Section: Related Workmentioning

confidence: 99%

Cross-Layer Autoencoder for Zero-Shot Learning

Zhang

Yang³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Zero-shot learning (ZSL) is the task of recognizing samples from their related classes which have never been seen during model training. ZSL is generally realized through learning a common embedding space where both high dimensional visual features and some pre-defined semantics can be mapped. However, this kind of solutions usually suffers from domain shift. In addition, the limitation and subjectivity of manual semantic information can also affect the classification results. To address these challenges, this paper proposes a novel end-to-end deep learning model called Cross-Layer Autoencoder (CLAE), which integrates different ways of semantic mapping and maintains reconstruction information. Besides, a regularized loss function is used to preserve local class manifolds. Extensive experiments for both traditional and generalized ZSL tasks are conducted on several benchmark datasets, and high effectiveness of the proposed method and its superiority over many previous researches are demonstrated. INDEX TERMS Zero-shot learning, cross-layer autoencoder, regularized loss function, class manifolds.

show abstract

Section: Related Workmentioning

confidence: 99%

Cross-Layer Autoencoder for Zero-Shot Learning

Zhang

Yang³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…How to undertake new tasks with a small amount of labeled data has become a new research hotspot [15,16]. Few-shot learning (FSL) provides a solution to this problem and meta-learning is a general framework to achieve the goal of few-shot learning [17][18][19][20].…”

Section: Introductionmentioning

confidence: 99%

Task-wise attention guided part complementary learning for few-shot image classification

Cheng

Lang

et al. 2021

Sci. China Inf. Sci.

View full text Add to dashboard Cite

A general framework to tackle the problem of few-shot learning is meta-learning, which aims to train a well-generalized meta-learner (or backbone network) to learn a base-learner for each future task with small training data. Although a lot of work has produced relatively good results, there are still some challenges for few-shot image classification. First, meta-learning is a learning problem over a collection of tasks and the meta-learner is usually shared among all tasks. To achieve image classification of novel classes in different tasks, it is needed to learn a base-learner for each task. Under the circumstances, how to make the base-learner specialized, and thus respond to different inputs in an extremely task-wise manner for different tasks is a big challenge at present. Second, classification network usually inclines to identify local regions from the most discriminative object parts rather than the whole objects for recognition, thereby resulting in incomplete feature representations. To address the first challenge, we propose a task-wise attention (TWA) module to guide the base-learner to extract task-specific image features. To address the second challenge, under the guidance of TWA, we propose a part complementary learning (PCL) module to extract and fuse the features of multiple complementary parts of target objects, and thus we can obtain more specific and complete information. In addition, the proposed TWA module and PCL module can be embedded into a unified network for end-to-end training. Extensive experiments on two commonly-used benchmark datasets and comparison with state-of-the-art methods demonstrate the effectiveness of our proposed method.

show abstract

“…For example, humans can recognize a zebra when they first see its appearance based only on the description that a zebra is a striped horse. The concept of zero-shot classification (ZSC) has been introduced to emulate this capability [4,5].…”

Section: Introductionmentioning

confidence: 99%

“…Early ZSC methods rely on semantically meaningful attributes to transfer knowledge [4,6]. The majority of these methods transfer cross-modal information [5,7] through the joint embedding of image visual features and attributes [8][9][10][11]. As intermediate representations, attributes share properties across multiple classes, indicating whether some predefined properties exist.…”

Section: Introductionmentioning

confidence: 99%

Triple discriminator generative adversarial network for zero-shot image classification

Yan

Wang

et al. 2021

Sci. China Inf. Sci.

Self Cite

View full text Add to dashboard Cite

One key challenge in zero-shot classification (ZSC) is the exploration of knowledge hidden in unseen classes. Generative methods such as generative adversarial networks (GANs) are typically employed to generate the visual information of unseen classes. However, the majority of these methods exploit global semantic features while neglecting the discriminative differences of local semantic features when synthesizing images, which may lead to sub-optimal results. In fact, local semantic information can provide more discriminative knowledge than global information can. To this end, this paper presents a new triple discriminator GAN for ZSC called TDGAN, which incorporates a text-reconstruction network into a dual discriminator GAN (D2GAN), allowing to realize cross-modal mapping from text descriptions to their visual representations. The text-reconstruction network focuses on key text descriptions for aligning semantic relationships to enable synthetic visual features to effectively represent images. Sharma-Mittal entropy is exploited in the loss function to make the distribution of synthetic classes be as close as possible to the distribution of real classes. The results of extensive experiments over the Caltech-UCSD Birds-2011 and North America Birds datasets demonstrate that the proposed TDGAN method consistently yields competitive performance compared to several state-of-the-art ZSC methods.

show abstract

Attribute-Guided Network for Cross-Modal Zero-Shot Hashing

Cited by 75 publications

References 38 publications

Cross-Layer Autoencoder for Zero-Shot Learning

Cross-Layer Autoencoder for Zero-Shot Learning

Task-wise attention guided part complementary learning for few-shot image classification

Triple discriminator generative adversarial network for zero-shot image classification

Contact Info

Product

Resources

About