Generalized Zero-Shot Learning via Over-Complete Distribution

Keshari, Rohit; Singh, Richa; Vatsa, Mayank

doi:10.1109/cvpr42600.2020.01331

Cited by 104 publications

(63 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed two HarS based inductive methods, HarS-WGAN and HarS-VAEGAN are compared with 18 stateof-the-art inductive methods: DEVISE [4], LATEM [17], SAE [49], DEM [22], LiGAN [28], ABP [30], TCN [24], OCD-GZSL [50], APNet [51], DE-VAE [52], LsrGAN [53], f-CSLWGAN [11], DASCN [27], DVBE [9], DAZLE [54], TF-VAEGAN [13], CF-GZSL [19], GCM-CF [37]. The proposed two HarST based transductive methods, HarST-DEM and HarST-WGAN are compared with 14 state-ofthe-art transductive methods: ALE-tran [16], GFZSL [55], DSRL [56], QFSL [40], GMN [57], f-VAEGAN-D2 [44], GXE [42], SABR-T [29], PREN [41], VSC [58], DTN [59], ADA [60], SDGN [45], TF-VAEGAN [13].…”

Section: Datasets and Comparative Methodsmentioning

confidence: 99%

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Liu¹,

Hu²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

This work is a systematical analysis on the so-called hard class problem in zero-shot learning (ZSL), that is, some unseen classes disproportionally affect the ZSL performances than others, as well as how to remedy the problem by detecting and exploiting hard classes. At first, we report our empirical finding that the hard class problem is a ubiquitous phenomenon and persists regardless of used specific methods in ZSL. Then, we find that high semantic affinity among unseen classes is a plausible underlying cause of hardness and design two metrics to detect hard classes. Finally, two frameworks are proposed to remedy the problem by detecting and exploiting hard classes, one under inductive setting, the other under transductive setting. The proposed frameworks could accommodate most existing ZSL methods to further significantly boost their performances with little efforts. Extensive experiments on three popular benchmarks demonstrate the benefits by identifying and exploiting the hard classes in ZSL.

show abstract

Section: Datasets and Comparative Methodsmentioning

confidence: 99%

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Liu¹,

Hu²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We selected recent state-of-the-art ZSL methods for comparison, which include methods without endto-end training such as PSR [4], RN [48], SP-AEN [9], IIR [7], TCN [24], E-PGN [62], DA-ZLE [22], f-CLSWGAN [56], cycle-CLSWGAN [12], CADA-VAE [45], OCD-CVAE [27], RFF-GZSL [17], IZF [46], and LsrGAN [51], where the last seven methods are feature generation based models, and end-to-end methods QFSL [47], LDF [33], SGMA [67], AREN [58], LF-GAA [35], DVBE [38], RGEN [59], and APN [60].…”

Section: Comparison With the State-of-the-artmentioning

confidence: 99%

Goal-Oriented Gaze Estimation for Zero-Shot Learning

Liu

Zhou

Bai

et al. 2021

2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

102

View full text Add to dashboard Cite

Zero-shot learning (ZSL) aims to recognize novel classes by transferring semantic knowledge from seen classes to unseen classes. Since semantic knowledge is built on attributes shared between different classes, which are highly local, strong prior for localization of object attribute is beneficial for visual-semantic embedding. Interestingly, when recognizing unseen images, human would also automatically gaze at regions with certain semantic clue. Therefore, we introduce a novel goal-oriented gaze estimation module (GEM) to improve the discriminative attribute localization based on the class-level attributes for ZSL. We aim to predict the actual human gaze location to get the visual attention regions for recognizing a novel object guided by attribute description. Specifically, the task-dependent attention is learned with the goal-oriented GEM, and the global image features are simultaneously optimized with the regression of local attribute features. Experiments on three ZSL benchmarks, i.e., CUB, SUN and AWA2, show the superiority or competitiveness of our proposed method against the state-of-the-art ZSL methods. The ablation analysis on real gaze data CUB-VWSW also validates the benefits and accuracy of our gaze estimation module. This work implies the promising benefits of collecting human gaze dataset and automatic gaze estimation algorithms on high-level computer vision tasks. The code is available at https://github.com/osierboy/GEM-ZSL.

show abstract

“…Due to the above methods often conduct mapping in a unidirectional manner, which tend to easily cause the domain shift problem. Therefore, the common-space based methods try to map different features into common spaces for connecting visual features with semantic features (Akata et al 2015b,a;Sung et al 2018;Keshari, Singh, and Vatsa 2020). CVAE (Mishra et al 2018) utilizes conditional VAE to encode the concatenation feature of visual and semantic embedding vectors.…”

Section: Related Workmentioning

confidence: 99%

“…CVAE (Mishra et al 2018) utilizes conditional VAE to encode the concatenation feature of visual and semantic embedding vectors. OCD (Keshari, Singh, and Vatsa 2020) designs an over-complete distribution of seen and unseen classes to enhance class separability. Although these methods extract the effective features on the common space, they often neglect explicitly aligning different modalities.…”

Section: Related Workmentioning

confidence: 99%

“…We compare our model with 15 state-ofthe-art (SOTA) models. Among them, SP-AEN (Chen et al 2018), AREN (Xie et al 2019), DAZLE (Huynh and Elhamifar 2020) are deep embedding based methods, cycle-CLSWGAN (Felix et al 2018), f-CLSWGAN (Xian et al 2018b), LiGAN (Li et al 2019), f-VAEGAN-D2 (Xian et al (Ni, Zhang, and Xie 2019), LsrGAN (Vyas, Venkateswara, and Panchanathan 2020), E-PGN (Yu et al 2020) are GAN based methods, and SE (Verma et al 2018), CVAE (Mishra et al 2018), CADA-VAE (Schönfeld et al 2019), OCD (Keshari, Singh, and Vatsa 2020), DE-VAE (Ma and Hu 2020), Disentangled-VAE (Li et al 2021)…”

Section: Implementation Detailsmentioning

confidence: 99%

See 1 more Smart Citation

Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification

Fang¹,

Zhu²,

Yuan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Learning a common latent embedding by aligning the latent spaces of cross-modal autoencoders is an effective strategy for Generalized Zero-Shot Classification (GZSC). However, due to the lack of fine-grained instance-wise annotations, it still easily suffer from the domain shift problem for the discrepancy between the visual representation of diversified images and the semantic representation of fixed attributes. In this paper, we propose an innovative autoencoder network by learning Aligned Cross-Modal Representations (dubbed ACMR) for GZSC. Specifically, we propose a novel Vision-Semantic Alignment (VSA) method to strengthen the alignment of cross-modal latent features on the latent subspaces guided by a learned classifier. In addition, we propose a novel Information Enhancement Module (IEM) to reduce the possibility of latent variables collapse meanwhile encouraging the discriminative ability of latent variables. Extensive experiments on publicly available datasets demonstrate the stateof-the-art performance of our method.

show abstract

Generalized Zero-Shot Learning via Over-Complete Distribution

Cited by 104 publications

References 20 publications

HardBoost: Boosting Zero-Shot Learning with Hard Classes

HardBoost: Boosting Zero-Shot Learning with Hard Classes

Goal-Oriented Gaze Estimation for Zero-Shot Learning

Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification

Contact Info

Product

Resources

About