Attentive Region Embedding Network for Zero-Shot Learning

Xie, Guo-Sen; Liu, Li; Jin, Xiao-Bo; Zhu, Fan; Zhang, Zheng; Qin, Jie; Yao, Yazhou; Shao, Ling

doi:10.1109/cvpr.2019.00961

Cited by 224 publications

(157 citation statements)

References 37 publications

Supporting

Mentioning

157

Contrasting

Order By: Relevance

“…Attention in ZSL The attention module has been applied for zero-shot learning for localization. Liu et al (2019c) introduces localization on the semantic information and Xie et al (2019) proposes localization on the visual feature. Zhu et al (2019c) further extends it to multi-localization guided by semantic.…”

Section: Related Workmentioning

confidence: 99%

Attribute Propagation Network for Graph Zero-Shot Learning

Liu

Zhou

Long

et al. 2020

AAAI

View full text Add to dashboard Cite

The goal of zero-shot learning (ZSL) is to train a model to classify samples of classes that were not seen during training. To address this challenging task, most ZSL methods relate unseen test classes to seen(training) classes via a pre-defined set of attributes that can describe all classes in the same semantic space, so the knowledge learned on the training classes can be adapted to unseen classes. In this paper, we aim to optimize the attribute space for ZSL by training a propagation mechanism to refine the semantic attributes of each class based on its neighbors and related classes on a graph of classes. We show that the propagated attributes can produce classifiers for zero-shot classes with significantly improved performance in different ZSL settings. The graph of classes is usually free or very cheap to acquire such as WordNet or ImageNet classes. When the graph is not provided, given pre-defined semantic embeddings of the classes, we can learn a mechanism to generate the graph in an end-to-end manner along with the propagation mechanism. However, this graph-aided technique has not been well-explored in the literature. In this paper, we introduce the “attribute propagation network (APNet)”, which is composed of 1) a graph propagation model generating attribute vector for each class and 2) a parameterized nearest neighbor (NN) classifier categorizing an image to the class with the nearest attribute vector to the image's embedding. For better generalization over unseen classes, different from previous methods, we adopt a meta-learning strategy to train the propagation mechanism and the similarity metric for the NN classifier on multiple sub-graphs, each associated with a classification task over a subset of training classes. In experiments with two zero-shot learning settings and five benchmark datasets, APNet achieves either compelling performance or new state-of-the-art results.

show abstract

Section: Related Workmentioning

confidence: 99%

Attribute Propagation Network for Graph Zero-Shot Learning

Liu

Zhou

Long

et al. 2020

AAAI

View full text Add to dashboard Cite

show abstract

“…In a case where all attributes form all-OR group, It becomes similar to ESZSL [137] and learns a bilinear compatibility function. AREN [190] uses attentive region embedding while learning the bilinear mapping to the semantic space in order to enhance the semantic transfer. ZSLPP [38] combines two networks VPDE-net for detecting bird parts from images and PZSC-net that trains a part-based Zero-Shot classifier from the noisy text of the Wikipedia.…”

Section: Label Embeddingmentioning

confidence: 99%

Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review

Rezaei

Shahidi

2020

SSRN Journal

View full text Add to dashboard Cite

The challenge of learning a new concept, object, or a new medical disease recognition without receiving any examples beforehand is called Zero-Shot Learning (ZSL). One of the major issues in deep learning based methodologies such as in Medical Imaging and other real-world applications is the requirement of large annotated datasets prepared by clinicians or experts to train the model. ZSL is known for having minimal human intervention by relying only on previously known or trained concepts plus currently existing auxiliary information. This is ever-growing research for the cases where we have very limited or no annotated datasets available and the detection = recognition system has human-like characteristics in learning new concepts. This makes the ZSL applicable in many real-world scenarios, from unknown object detection in autonomous vehicles to medical imaging and unforeseen diseases such as COVID-19 Chest X-Ray (CXR) based diagnosis. In this review paper, we introduce a novel and broaden solution called Few = one-shot learning, and present the definition of the ZSL problem as an extreme case of the few-shot learning. We review over fundamentals and the challenging steps of Zero-Shot Learning, including state-of-the-art categories of solutions, as well as our recommended solution, motivations behind each approach, their advantages over each category to guide both clinicians and AI researchers to proceed with the best techniques and practices based on their applications. Inspired from different settings and extensions, we then review through different datasets inducing medical and non-medical images, the variety of splits, and the evaluation protocols proposed so far. Finally, we discuss the recent applications and future directions of ZSL. We aim to convey a useful intuition through this paper towards the goal of handling complex learning tasks more similar to the way humans learn. We mainly focus on two applications in the current modern yet challenging era: coping with an early and fast diagnosis of COVID-19 cases, and also encouraging the readers to develop other similar AI-based automated detection = recognition systems using ZSL.

show abstract

“…A multi-attention loss encourages compact and diverse attention distribution by applying geometric constraints over attention maps. The AREN model [35] discovers multiple semantic parts of images guided by an attention mechanism and the compatibility loss. The model is also coupled with a paralleled network branch to guarantee more stable semantic transfer from the perspective of second-order collaboration.…”

Section: Attention Mechanisms In Zslmentioning

confidence: 99%

“…Conventionally, training a traditional classi cation model requires at least some data samples for all target classes, and deep learning models signi cantly amplify this issue. Collecting training instances of every class is not always easy, especially in ne-grained image classi cation [19,20,36], and therefore much attention has been given to zero-shot learning (ZSL) algorithms as a solution [3,5,10,17,18,21,22,25,27,35,38]. ZSL expands the classi ers beyond the seen classes with abundant data to unseen classes without enough image samples.…”

Section: Introductionmentioning

confidence: 99%

Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches

Chen

Wang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Zero-shot learning (ZSL) is commonly used to address the very pervasive problem of predicting unseen classes in ne-grained image classi cation and other tasks. One family of solutions is to learn synthesised unseen visual samples produced by generative models from auxiliary semantic information, such as natural language descriptions. However, for most of these models, performance su ers from noise in the form of irrelevant image backgrounds. Further, most methods do not allocate a calculated weight to each semantic patch. Yet, in the real world, the discriminative power of features can be quanti ed and directly leveraged to improve accuracy and reduce computational complexity. To address these issues, we propose a novel framework called multi-patch generative adversarial nets (MPGAN) that synthesises local patch features and labels unseen classes with a novel weighted voting strategy. The process begins by generating discriminative visual features from noisy text descriptions for a set of prede ned local patches using multiple specialist generative models. The features synthesised from each patch for unseen classes are then used to construct an ensemble of diverse supervised classi ers, each corresponding to one local patch. A voting strategy averages the probability distributions output from the classi ers and, given that some patches are more discriminative than others, a discrimination-based attention mechanism helps to weight each patch accordingly. Extensive experiments show that MPGAN has signi cantly greater accuracy than state-of-the-art methods. CCS CONCEPTS • Computing methodologies → Object recognition.

show abstract

Attentive Region Embedding Network for Zero-Shot Learning

Cited by 224 publications

References 37 publications

Attribute Propagation Network for Graph Zero-Shot Learning

Attribute Propagation Network for Graph Zero-Shot Learning

Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review

Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches

Contact Info

Product

Resources

About