Generalized Zero Shot Learning via Synthesis Pseudo Features

Li, Chuanlong; Ye, Xiufen; Yang, Haibo; Ying, Han; Li, Xiang; Jia, Yunpeng

doi:10.1109/access.2019.2925093

Cited by 13 publications

(19 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In TZSL, the search space is C u , i.e., Y = C u . Our algorithm is compared with 10 recently proposed baseline algorithms for TZSL task, including DEVISE [7], SJE [44], ALE [11], SYNC [18], SAE [13], DEM [14], GFZSL [45], LESAE [46], PSR [24], and RAS-GAN [26]. To show the effectiveness of the proposed, we compared the simulated results with 10 other algorithms.…”

Section: B Results On Traditional Zero-shot Learning (Tzsl) Tasksmentioning

confidence: 99%

“…Typical synthesisbased methods include the followings. SPF [24] calculates the semantic attributes correlation between the seen and unseen classes to find out several seen classes that most similar to the unseen classes. And then it uses the visual features of the seen classes to multiply the similarity coefficient to synthesize the visual features of the unseen classes.…”

Section: Related Work a Zero-shot Learningmentioning

confidence: 99%

See 1 more Smart Citation

Cooperative Coupled Generative Networks for Generalized Zero-Shot Learning

et al. 2020

View full text Add to dashboard Cite

Compared with zero-shot learning (ZSL), the generalized zero-shot learning (GZSL) is more challenging since its test samples are taken from both seen and unseen classes. Most previous mapping-based methods perform well on ZSL, while their performance degrades on GZSL. To solve this problem, inspired by the ensemble learning, this paper proposes a model with cooperative coupled generative networks (CCGN). Firstly, to alleviate the hubness problem, the reverse visual feature space is taken as the embedding space, with the mapping achieved by a visual feature center generation network. To learn a proper visual representation of each class, we propose a coupled of generative networks, which cooperate with each other to synthesize a visual feature center template of the class. Secondly, to improve the generative ability of the coupled networks, we further employ a deeper network to generate. Meanwhile, to alleviate loss semantic information problem caused by multiple network layers, a residual module is employed. Thirdly, to mitigate overfitting and to increase scalability, an adversarial network is introduced to discriminate the generation of visual feature centers. Finally, a reconstruction network, which reverses the generation process, is employed to restrict the structural correlation between the generated visual feature center and the original semantic representation of each class. Extensive experiments on five benchmark datasets (AWA1, AWA2, CUB, SUN, APY) demonstrate that the proposed algorithm yields satisfactory results, as compared with the state-of-the-art methods.

show abstract

Section: B Results On Traditional Zero-shot Learning (Tzsl) Tasksmentioning

confidence: 99%

Section: Related Work a Zero-shot Learningmentioning

confidence: 99%

Cooperative Coupled Generative Networks for Generalized Zero-Shot Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…MFMR [ 193 ] exploits the manifold structure of test data with a joint prediction scheme to avoid domain shift [ 138 ]. use entropy minimisation in optimisation [ 86 ]. preserve the semantic similarity structure in seen and unseen classes to avoid the domain-shift occurrence [ 87 ].…”

Section: Discussionmentioning

confidence: 99%

“…ZSL models can be seen from two points of views in terms of training and test phase: Classic ZSL and Generalised ZSL (GZSL) settings. In the classic ZSL settings, the model only detects the presence of new classes at the test phase, while in GZSL settings, the model predicts both unseen and seen classes at the test time; hence, GZSL is more applicable for real-world scenarios [ 94 ], [ 75 ], [ 210 ], [ 86 ], [ 145 ]. The same idea can be applied to FSL to train in the generalised model, called generalised few-shot learning (GFSL) that detects both known and novel classes at the test time.…”

Section: Zsl Test and Training Phasesmentioning

confidence: 99%

“…It also introduces Gradient Matching (GM) loss to improve the quality of the synthesised features. In order to synthesise unseen features, SPF-GZSL [ 86 ] selects similar instances and combines them to form pseudo features using a centre loss function [ 182 ]. In Don’t Even Look Once (DELO) by [ 209 ] a detection algorithm is conducted to synthesise unseen visual features to gain high confidence predictions for unseen concepts while maintaining low confidence for backgrounds with vanilla detectors.…”

Section: Zsl Data Embedding Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Zero-shot learning and its applications from autonomous vehicles to COVID-19 diagnosis: A review

Rezaei

Shahidi

2020

Intelligence-Based Medicine

View full text Add to dashboard Cite

The challenge of learning a new concept, object, or a new medical disease recognition without receiving any examples beforehand is called Zero-Shot Learning (ZSL). One of the major issues in deep learning based methodologies such as in Medical Imaging and other real-world applications is the requirement of large annotated datasets prepared by clinicians or experts to train the model. ZSL is known for having minimal human intervention by relying only on previously known or trained concepts plus currently existing auxiliary information. This is ever-growing research for the cases where we have very limited or no annotated datasets available and the detection recognition system has human-like characteristics in learning new concepts. This makes the ZSL applicable in many real-world scenarios, from unknown object detection in autonomous vehicles to medical imaging and unforeseen diseases such as COVID-19 Chest X-Ray (CXR) based diagnosis. In this review paper, we introduce a novel and broaden solution called Few one-shot learning, and present the definition of the ZSL problem as an extreme case of the few-shot learning. We review over fundamentals and the challenging steps of Zero-Shot Learning, including state-of-the-art categories of solutions, as well as our recommended solution, motivations behind each approach, their advantages over each category to guide both clinicians and AI researchers to proceed with the best techniques and practices based on their applications. Inspired from different settings and extensions, we then review through different datasets inducing medical and non-medical images, the variety of splits, and the evaluation protocols proposed so far. Finally, we discuss the recent applications and future directions of ZSL. We aim to convey a useful intuition through this paper towards the goal of handling complex learning tasks more similar to the way humans learn. We mainly focus on two applications in the current modern yet challenging era: coping with an early and fast diagnosis of COVID-19 cases, and also encouraging the readers to develop other similar AI-based automated detection recognition systems using ZSL.

show abstract

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Zhao

Liu

et al. 2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Zero‐shot learning (ZSL) addresses the novel object recognition problem by leveraging semantic embedding to transfer knowledge from seen categories to unseen categories. Generative ZSL models synthesize the visual features of unseen classes and convert ZSL task into a classical supervised learning problem. These generative ZSL models are trained by using the seen classes. Although promising progress has been achieved in the ZSL and generalized zero‐shot learning (GZSL) tasks. The existing approaches still suffer from a strong bias problem between unseen and seen classes, where unseen objects in the target domain tend to be recognized as seen classes in the source domain. To deal with the problem, we propose a novel named semantic consistent Wasserstein generative adversarial network (scWGAN), which uses a semantic reconstructor to reconstruct semantic embeddings from generated visual features by incorporating a novel Semantic Consistent Loss noted L rec. The Semantic Consistent Loss guides our proposed scWGAN to generate visual features that mirror the semantic relationships between seen and unseen classes. We also introduce a visual classifier to constrain visual feature generator. Extensive experiments show that the proposed approach is superior to previous state‐of‐the‐art works under both traditional ZSL and challenging GZSL settings on six popular data sets AWA1, AWA2, CUB, APY, and SUN.

show abstract

Generalized Zero Shot Learning via Synthesis Pseudo Features

Cited by 13 publications

References 41 publications

Cooperative Coupled Generative Networks for Generalized Zero-Shot Learning

Cooperative Coupled Generative Networks for Generalized Zero-Shot Learning

Zero-shot learning and its applications from autonomous vehicles to COVID-19 diagnosis: A review

Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

Contact Info

Product

Resources

About