Generative Model with Semantic Embedding and Integrated Classifier for Generalized Zero-Shot Learning

Pambala, Ayyappa Kumar; Dutta, Titir; Biswas, Soma

doi:10.1109/wacv45572.2020.9093625

Cited by 23 publications

(13 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This means the proposed method can generate well discriminative features for improved dual-domain classification. Both versions of BMCoGAN achieve improved performance than ISE-GAN [44] and IBZSL [45], and verifies better discriminative learning, which reduces bias towards seen classes.…”

Section: A Results and Analysismentioning

confidence: 60%

Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning

Shermin¹,

Teng²,

Sohel³

et al. 2020

Preprint

View full text Add to dashboard Cite

Bidirectional mapping-based generative models have achieved remarkable performance for the generalized zero-shot learning (GZSL) recognition by learning to construct visual features from class semantics and reconstruct class semantics back from generated visual features. The performance of these models relies on the quality of synthesized features. This depends on the ability of the model to capture the underlying seen data distribution by relating semantic-visual spaces, learning discriminative information, and re-purposing the learned distribution to recognize unseen data. This means learning the seen-unseen domains joint distribution is crucial for GZSL tasks. However, existing models only learn the underlying distribution of the seen domain as unseen data is inaccessible. In this work, we propose to utilize the available unseen class semantics along with seen class semantics and learn dual-domain joint distribution through a strong visual-semantic coupling. Therefore, we propose a bidirectional mapping coupled generative adversarial network (BMCoGAN) by extending the coupled generative adversarial network (CoGAN) into a dual-domain learning bidirectional mapping model. We further integrate a Wasserstein generative adversarial optimization to supervise the joint distribution learning. For retaining distinctive information in the synthesized visual space and reducing bias towards seen classes, we design an optimization, which pushes synthesized seen features towards real seen features and pulls synthesized unseen features away from real seen features. We evaluate BMCoGAN on several benchmark datasets against contemporary methods and show its superior performance. Also, we present ablative analysis to demonstrate the importance of different components in BMCoGAN.

show abstract

Section: A Results and Analysismentioning

confidence: 60%

Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning

Shermin¹,

Teng²,

Sohel³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…ZSL is usually provided with an auxiliary set of class attributes to describe each seen-and unseen-class [31,63]. Therefore, ZSL can be approached by either inferring a sample's attribute and finding the closest match in the attribute space [16,1,13,25], or generating features using the attributes and matching in the feature space [65,34,41,45]. OSR is a more challenging open environment setting with no information on the unseen-classes [51,52], and the goal is to build a classifier for seen-classes that additionally rejects unseen-class samples as outliers.…”

Section: Related Workmentioning

confidence: 99%

Counterfactual Zero-Shot and Open-Set Visual Recognition

Yue¹,

Wang²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR), whose common challenge is generalizing to the unseen-classes by only training on the seen-classes. Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution, which causes severe recognition rate imbalance between the seen-class (high) and unseen-class (low). We show that the key reason is that the generation is not Counterfactual Faithful, and thus we propose a faithful one, whose generation is from the sample-specific counterfactual question: What would the sample look like, if we set its class attribute to a certain class, while keeping its sample attribute unchanged? Thanks to the faithfulness, we can apply the Consistency Rule to perform unseen/seen binary classification, by asking: Would its counterfactual still look like itself? If "yes", the sample is from a certain class, and "no" otherwise. Through extensive experiments on ZSL and OSR, we demonstrate that our framework effectively mitigates the seen/unseen imbalance and hence significantly improves the overall performance. Note that this framework is orthogonal to existing methods, thus, it can serve as a new baseline to evaluate how ZSL/OSR models generalize.Codes are available at https://github.com/yuezhongqi/gcm-cf.* Equal contribution 1 Conventional ZSL [31] only evaluates the recognition on the unseenclasses. We refer to ZSL as a more challenging setting: Generalized ZSL [12], which is evaluated on the both seen-and unseen-classes.

show abstract

“…Note that, this information can be easily obtained given just the class names in D base , and does not require any additional information. Attributes are widely used in applications like ZSL [21,22] , but it has been relatively less explored for few-shot learning.…”

Section: Problem Definitionmentioning

confidence: 99%

SML: Semantic meta-learning for few-shot semantic segmentation☆

Pambala

Dutta

Biswas

2021

Pattern Recognition Letters

Self Cite

View full text Add to dashboard Cite

The significant amount of training data required for training Convolutional Neural Networks has become a bottleneck for applications like semantic segmentation. Few-shot semantic segmentation algorithms address this problem, with an aim to achieve good performance in the low-data regime, with few annotated training images. Recent approaches based on class-prototypes computed from available training data have achieved immense success for this task. In this work, we propose a novel meta-learning framework, Semantic Meta-Learning (SML), which incorporates class level semantic descriptions in the generated prototypes for this problem. In addition, we propose to use the well-established technique, ridge regression, to not only bring in the class-level semantic information, but also to effectively utilise the information available from multiple images present in the training data for prototype computation. This has a simple closed-form solution, and thus can be implemented easily and efficiently. Extensive experiments on the benchmark PASCAL-5i dataset under different experimental settings demonstrate the effectiveness of the proposed framework.

show abstract

Generative Model with Semantic Embedding and Integrated Classifier for Generalized Zero-Shot Learning

Cited by 23 publications

References 22 publications

Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning

Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning

Counterfactual Zero-Shot and Open-Set Visual Recognition

SML: Semantic meta-learning for few-shot semantic segmentation☆

Contact Info

Product

Resources

About