Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning

Li, Aoxue; Lu, Zhiwu; Guan, Jiechao; Xiang, Tao; Wang, Liwei; Wen, Ji-Rong

doi:10.1007/s11263-020-01342-x

Cited by 17 publications

(8 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specially, Changpinyo et al [27] considered both class attributes and word embeddings to calculate weighted edges between entities. Different from the above methods that use some auxiliary information for building KG edges, Zhao et al [224] and Geng et al [62,63] modeled the class attributes as additional KG entities and connected them to the entities of the classes; while Li et al [100,101] generated new superclasses of the seen and unseen classes by clustering of the class names, so as to constructing class hierarchies for augmenting ZSL and FSL. Domain knowledge, which is often in the form of heuristics and logic rules, has also be used to construct task-specific KGs.…”

Section: 22mentioning

confidence: 99%

“…Input Mapping [34,84,100,102,103,105,114,132] Class Mapping [1,2,27,127,158] Joint Mapping [38,41,70,83,114,149,152] Data Augmentation…”

Section: Mapping-basedmentioning

confidence: 99%

“…Chen et al [34] adopted a typical ZSL method named Semantic Autoencoder (SAE) [92], and used ontology embedding for the initial class encoding 𝒚. They used a linear encoder to map the input 𝒙 to the class's semantic embedding space as 𝒙 ′ , where this encoder is learned on D 𝑡𝑟 by minimizing a distance loss between 𝒙 ′ and 𝒚 and a reconstruction loss when 𝒙 ′ is mapped back to 𝒙. Li et al [100] proposed to use a Long-Short-Term-Memory (LSTM) network to model the class hierarchy and map the image features learned by a CNN to hierarchical classes. In detail, one LSTM module uses the output of the subclass-level fully connected layer of CNN as input, and another LSTM module uses the output of the above LSTM module as well as the output of the superclass-level fully connected layer of CNN as input and predicts the probability distribution of superclasses.…”

Section: Mapping-based Paradigmmentioning

confidence: 99%

“…Input Mapping [85,114,125] Class Mapping [100] Joint Mapping [1,2,101,114,149,165,204,215,218,224] Data Augmentation…”

Section: Mapping-basedmentioning

confidence: 99%

“…However, such extension has been rarely investigated. The only relevant one is by Li et al[100], which projects the embeddings of hierarchical classes into the space of image features learned by CNN to support the zero-shot image classification, and extends to the few-shot setting with almost no modification. The mapping function learning does not use the samples of seen classes, but in prediction, the average of the CNN features of the few-shot samples as well as the class vector after mapping are used in searching the class label of a testing sample of unseen classes.5.1.3 Joint Mapping.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Zero-shot and Few-shot Learning with Knowledge Graphs: A Comprehensive Survey

Chen¹,

Geng²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Machine learning methods especially deep neural networks have achieved great success but many of them often rely on a number of labeled samples for training. In real-world applications, we often need to address sample shortage due to e.g., dynamic contexts with emerging prediction targets and costly sample annotation. Therefore, low-resource learning, which aims to learn robust prediction models with no enough resources (especially training samples), is now being widely investigated. Among all the low-resource learning studies, many prefer to utilize some auxiliary information in the form of Knowledge Graph (KG), which is becoming more and more popular for knowledge representation, to reduce the reliance on labeled samples. In this survey, we very comprehensively reviewed over 90 papers about KG-aware research for two major low-resource learning settings -zero-shot learning (ZSL) where new classes for prediction have never appeared in training, and few-shot learning (FSL) where new classes for prediction have only a small number of labeled samples that are available. We first introduced the KGs used in ZSL and FSL studies as well as the existing and potential KG construction solutions, and then systematically categorized and summarized KG-aware ZSL and FSL methods, dividing them into different paradigms such as the mapping-based, the data augmentation, the propagation-based and the optimization-based. We next presented different applications, including not only KG augmented prediction tasks in Computer Vision and Natural Language Processing (e.g., image classification, visual question answering, text classification and knowledge extraction), but also tasks for KG curation (e.g., inductive KG completion), and some typical evaluation resources for each task. We eventually discussed some challenges and future directions on aspects such as new learning and reasoning paradigms, and the construction of high quality KGs.

show abstract