Hierarchical Disentanglement of Discriminative Latent Features for Zero-Shot Learning

Tong, Bin; Wang, Chao; Klinkigt, Martin; Kobayashi, Yoshiyuki; Nonaka, Yuuichi

doi:10.1109/cvpr.2019.01173

Cited by 57 publications

(41 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…iii) In particular, for the two coarse-grained datasets (aPY and AWA2), our CPL achieves 0.5% and 5.6% significant improvements over the strongest competitors, showing its great advantage in coarse-grained object recognition problems. iv) Although DLFZRL [39] performs better than our CPL with 1.4% on CUB, it still can be observed that our CPL takes more advantage on most cases.…”

Section: Standard Zslmentioning

confidence: 77%

“…By contrast, our framework needs not to predefine these relations. Third, a recent work DLFZRL [39] aims to learn discriminative and generalizable representations from image features and then improves the performance of existing methods. In particular, they utilized DEVISE [11] as the embedding function and obtained very competitive results on four benchmark datasets.…”

Section: Standard Zslmentioning

confidence: 99%

See 1 more Smart Citation

Convolutional prototype learning for zero-shot recognition

Liu

Zhang

Zhu

et al. 2020

Image and Vision Computing

View full text Add to dashboard Cite

Zero-shot learning (ZSL) has received increasing attention in recent years especially in areas of fine-grained object recognition, retrieval, and image captioning. The key to ZSL is to transfer knowledge from the seen to the unseen classes via auxiliary class attribute vectors. However, the popularly learned projection functions in previous works cannot generalize well since they assume the distribution consistency between seen and unseen domains at sample-level. Besides, the provided non-visual and unique class attributes can significantly degrade the recognition performance in semantic space. In this paper, we propose a simple yet effective convolutional prototype learning (CPL) framework for zero-shot recognition. By assuming distribution consistency at task-level, our CPL is capable of transferring knowledge smoothly to recognize unseen samples. Furthermore, inside each task, discriminative visual prototypes are learned via a distance based training mechanism. Consequently, we can perform recognition in visual space, instead of semantic space. An extensive group of experiments are then carefully designed and presented, demonstrating that CPL obtains more favorable effectiveness, over currently available alternatives under various settings.

show abstract

Section: Standard Zslmentioning

confidence: 77%

Section: Standard Zslmentioning

confidence: 99%

Convolutional prototype learning for zero-shot recognition

Liu

Zhang

Zhu

et al. 2020

Image and Vision Computing

View full text Add to dashboard Cite

show abstract

“…There have been previous efforts to extract the semantic feature from the image feature (Tong et al 2019;Han, Fu, and Yang 2020;Li et al 2021;Chen et al 2021). While our approach seems to be a bit similar to (Li et al 2021) and (Chen et al 2021) in the sense that the autoencoder-based image feature decomposition method is used for the semantic feature extraction, our work is dearly distinct from those works in two respects.…”

Section: Comparison With Conventional Approachesmentioning

confidence: 99%

“…Datasets In our experiments, we evaluate the performance of our model using four benchmark datasets: AwA1, AwA2, 1 We recall that {z In dividing the total classes into seen and unseen classes, we adopt the conventional dataset split presented in (Xian, Schiele, and Akata 2017).…”

Section: Experiments Experimental Setupmentioning

confidence: 99%

Semantic Feature Extraction for Generalized Zero-shot Learning

Kim¹,

Shim²,

Shim³

2021

Preprint

View full text Add to dashboard Cite

Generalized zero-shot learning (GZSL) is a technique to train a deep learning model to identify unseen classes using the attribute. In this paper, we put forth a new GZSL technique that improves the GZSL classification performance greatly. Key idea of the proposed approach, henceforth referred to as semantic feature extraction-based GZSL (SE-GZSL), is to use the semantic feature containing only attribute-related information in learning the relationship between the image and the attribute. In doing so, we can remove the interference, if any, caused by the attribute-irrelevant information contained in the image feature. To train a network extracting the semantic feature, we present two novel loss functions, 1) mutual information-based loss to capture all the attribute-related information in the image feature and 2) similarity-based loss to remove unwanted attribute-irrelevant information. From extensive experiments using various datasets, we show that the proposed SE-GZSL technique outperforms conventional GZSL approaches by a large margin.

show abstract

“…Although remarkable results were obtained by these unsupervised DRL methods using toy datasets such as dSprites [22] and 3D Shapes [23], there is no guarantee that each latent variable corresponds to a single semantically meaningful factor of variation without any inductive bias [10], [24], [25]. Hence, recent DRL studies have focused on introduction to a model of an explicit prior that imposes constraints or regulariza-tions based on the underlying structure of complicated realworld images [26], [27], such as translation and rotation [2], [28], hierarchical features [8], [9], [29] and domain-specific knowledge [10].…”

Section: Introductionmentioning

confidence: 99%

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior

et al. 2021

View full text Add to dashboard Cite

We propose a novel method that can learn easy-to-interpret latent representations in realworld image datasets using a VAE-based model by splitting an image into several disjoint regions. Our method performs object-wise disentanglement by exploiting image segmentation and alpha compositing. With remarkable results obtained by unsupervised disentanglement methods for toy datasets, recent studies have tackled challenging disentanglement for real-world image datasets. However, these methods involve deviations from the standard VAE architecture, which has favorable disentanglement properties. Thus, for disentanglement in images of real-world image datasets with preservation of the VAE backbone, we designed an encoder and a decoder that embed an image into disjoint sets of latent variables corresponding to objects. The encoder includes a pre-trained image segmentation network, which allows our model to focus only on representation learning while adopting image segmentation as an inductive bias. Evaluations using real-world image datasets, CelebA and Stanford Cars, showed that our method achieves improved disentanglement and transferability.INDEX TERMS Alpha blend, disentanglement, image segmentation, real-world image, representation learning.

show abstract

Hierarchical Disentanglement of Discriminative Latent Features for Zero-Shot Learning

Cited by 57 publications

References 18 publications

Convolutional prototype learning for zero-shot recognition

Convolutional prototype learning for zero-shot recognition

Semantic Feature Extraction for Generalized Zero-shot Learning

Disentangled Representation Learning in Real-World Image Datasets via Image Segmentation Prior

Contact Info

Product

Resources

About