Self-Attention Relation Network for Few-Shot Learning

Hui, Binyuan; Zhu, Pengfei; Hu, Qinghua; Wang, Qilong

doi:10.1109/icmew.2019.00041

Cited by 37 publications

(14 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate the proposed MCRNet for few-shot classification tasks on the unseen meta-testing set and compare it with the state-of-the-art methods. 4CONV 58.9 ± 1.9 71.5 ± 1.0 --R2D2 [21] 4CONV 65.3 ± 0.2 79.4 ± 0.1 --Fine-tuning [32] ResNet-12 64.66 ± 0.73 82.13 ± 0.50 37.52 ± 0.53 55.39 ± 0.57 TADAM [22] ResNet-12 --40.1 ± 0.4 56.1 ± 0.4 MTL [8] ResNet-12 ¶ --43.6 ± 1.8 55.4 ± 0.9 Baseline-RR [29] ResNet [18] ResNet-101 ¶ 51.62 ± 0.31 66.16 ± 0.51 DN4 [33] ResNet-12 54.37 ± 0.36 74.44 ± 0.29 SNAIL [26] ResNet-12 55.71 ± 0.99 68.88 ± 0.92 Fine-tuning [32] ResNet-12 56.67 ± 0.62 74.80 ± 0.51 TADAM [22] ResNet-12 ¶ 58.50 ± 0.30 76.70 ± 0.30 CAML [34] ResNet-12 59.23 ± 0.99 72.35 ± 0.71 TPN [35] ResNet-12 59.46 75.65 wDAE-GNN [36] WRN-28-10 61.07 ± 0.15 76.75 ± 0.11 MTL [8] ResNet-12 ¶ 61.…”

Section: Methodsmentioning

confidence: 99%

“…Unfortunately, these methods choose a less expressive architecture as a feature extractor, like 4CONV [13], to avoid the representation deficiency in a high-dimensional parameter space. [29] tries to propose better base-learners and [18] interpolates self-attention relation network. These two extract features directly via more powerful architecture, ResNet-12 [22] or ResNet-101, but neither of them addresses the deficiency mentioned.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, a number of studies [13]- [17] in meta-learning seek to represent features directly with less expressive feature extractors. These methods generally alleviate the tendency of over-fitting in few-shot setting and the difficulty of representing features [8], [18], caused by the representation deficiency, but at the expense of greater representation or expressiveness. Furthermore, the inherent task ambiguity [16] in few-shot learning also leads to difficulties when scaling to high-dimensional data.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Complementing Representation Deficiency in Few-shot Image Classification: A Meta-Learning Approach

Zhong

Huang

et al. 2020

Preprint

View full text Add to dashboard Cite

Few-shot learning is a challenging problem that has attracted more and more attention recently since abundant training samples are difficult to obtain in practical applications. Meta-learning has been proposed to address this issue, which focuses on quickly adapting a predictor as a base-learner to new tasks, given limited labeled samples. However, a critical challenge for meta-learning is the representation deficiency since it is hard to discover common information from a small number of training samples or even one, as is the representation of key features from such little information. As a result, a meta-learner cannot be trained well in a high-dimensional parameter space to generalize to new tasks. Existing methods mostly resort to extracting less expressive features so as to avoid the representation deficiency. Aiming at learning better representations, we propose a metalearning approach with complemented representations network (MCRNet) for few-shot image classification. In particular, we embed a latent space, where latent codes are reconstructed with extra representation information to complement the representation deficiency. Furthermore, the latent space is established with variational inference, collaborating well with different baselearners, and can be extended to other models. Finally, our endto-end framework achieves the state-of-the-art performance in image classification on three standard few-shot learning datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Complementing Representation Deficiency in Few-shot Image Classification: A Meta-Learning Approach

Zhong

Huang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…SoSN [131] chose to perform the relation computation on the secondorder representation of feature maps. SARN [132] introduced a self-attention mechanism into Relation Net for capturing non-local features and enhancing representation.…”

Section: Relation Net and Its Variantsmentioning

confidence: 99%

Learning from Very Few Samples: A Survey

Lu¹,

Gong²,

Ye³

et al. 2020

Preprint

View full text Add to dashboard Cite

Few sample learning (FSL) is significant and challenging in the field of machine learning. The capability of learning and generalizing from very few samples successfully is a noticeable demarcation separating artificial intelligence and human intelligence since humans can readily establish their cognition to novelty from just a single or a handful of examples whereas machine learning algorithms typically entail hundreds or thousands of supervised samples to guarantee generalization ability. Despite the long history dated back to the early 2000s and the widespread attention in recent years with booming deep learning technologies, little surveys or reviews for FSL are available until now. In this context, we extensively review 300+ papers of FSL spanning from the 2000s to 2019 and provide a timely and comprehensive survey for FSL. In this survey, we review the evolution history as well as the current progress on FSL, categorize FSL approaches into the generative model based and discriminative model based kinds in principle, and emphasize particularly on the meta learning based FSL approaches. We also summarize several recently emerging extensional topics of FSL and review the latest advances on these topics. Furthermore, we highlight the important FSL applications covering many research hotspots in computer vision, natural language processing, audio and speech, reinforcement learning and robotic, data analysis, etc. Finally, we conclude the survey with a discussion on promising trends in the hope of providing guidance and insights to follow-up researches.

show abstract

“…Zhang et al [24] improve the relation network by using three-dimensional tensor features of different levels extracted by the embedded network. Hui et al [25] propose a self-attention relation network including an embedding module, an attention module, and a relation module. Besides, similarity measurement is also an important part of the meta-metric learning model, and a lot of work appeared around this aspect.…”

Section: Related Workmentioning

confidence: 99%

Comparative Analysis on Classical Meta-Metric Models for Few-Shot Learning

Yang

Liu

2020

IEEE Access

View full text Add to dashboard Cite

Few-shot learning are methods and scenarios learned from a small amount of labeled data. While recent meta-metric learning methods have made significant progress, there are still questions about what is the key point of these methods and how they work. To address these problems, in this paper, we evaluate the effects of different parts in classical models. To be specific, we 1) use four typical networks AlexNet, VGG16, GoogLeNet, and ResNet50 to replace the original feature extraction part in the Matching Network, Prototypical Network, and Relation Network, and compare the best results with 17 state-of-the-art meta-metric learning algorithms. 2) fix the feature extraction part of the Matching Network, Prototypical Network and Relation Network, and change the similarity measurement part of each into L1, L2, Cosine. 3) conduct above three models on datasets of different granularity. The experimental results show that for all models evaluated, the addition of non-pretrained networks will make the classification results worse, which shows that it is easy to overfit when using deep networks for few-shot learning. Changes in similarity measurement methods have a significant impact on results, which shows the importance to choose a suitable measurement. Moreover, there are differences in performance on different granularity datasets.

show abstract

Self-Attention Relation Network for Few-Shot Learning

Cited by 37 publications

References 4 publications

Complementing Representation Deficiency in Few-shot Image Classification: A Meta-Learning Approach

Complementing Representation Deficiency in Few-shot Image Classification: A Meta-Learning Approach

Learning from Very Few Samples: A Survey

Comparative Analysis on Classical Meta-Metric Models for Few-Shot Learning

Contact Info

Product

Resources

About