Subspace Adaptation Prior for Few-Shot Learning

Huisman, Mike; Plaat, Aske; van Rijn, Jan N.

doi:10.1007/s10994-023-06393-y

Cited by 2 publications

(1 citation statement)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Few-shot classification learning 11,12 aims to classify the unlabeled query samples using a small number of support samples. Few-shot classification learning uses meta-learning to learn and propagate transferable knowledge in a task set to improve generalization ability.…”

Section: Introductionmentioning

confidence: 99%

Cross-channel spatial attention network for few-shot classification

Jia,

Wang,

Zhu

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

.Few-shot image classification learns from the limited samples of known categories, with the purpose of identifying new categories of samples. However, due to the under-representation of the class prototype generated by support samples for few-shot image classification, it leads to large classification errors. Therefore, we propose the multi-loss dual prototype network. First, the class prototype of the support samples and the class prototype of the query samples are constructed by the support samples or query samples, respectively. Second, the local loss of self-calibration (SC) is obtained by calculating the difference between the class prototype features of the support samples and the support sample features. The local loss of mutual-calibration (MC) is obtained by computing the difference between the class prototype features of the query sample and the support sample features, as well as the difference between the class prototype features of the support samples and the query sample features. Then, the global loss of the support samples and the query sample classification is calculated, and the local and global losses are weighted as the total loss to train the network. The total loss includes both SC and MC, which enables the network to make full use of the sample feature information. To adaptively enhance the features of the target region, we propose a cross-channel spatial attention model that generates class prototype features that are more consistent with the feature distribution of the class. Finally, we improve the transductive inference algorithm by assigning weights to the unlabeled query samples using cosine similarity and expanding the weighted query samples into the support set through iteration to further enrich the class prototype features. Experimental results show that the proposed method is effective.

show abstract

Section: Introductionmentioning

confidence: 99%

Cross-channel spatial attention network for few-shot classification

Jia,

Wang,

Zhu

et al. 2023

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

Are LSTMs good few-shot learners?

Huisman,

Moerland,

Plaat

et al. 2023

Mach Learn

Self Cite

View full text Add to dashboard Cite

Deep learning requires large amounts of data to learn new tasks well, limiting its applicability to domains where such data is available. Meta-learning overcomes this limitation by learning how to learn. Hochreiter et al. (International conference on artificial neural networks, Springer, 2001) showed that an LSTM trained with backpropagation across different tasks is capable of meta-learning. Despite promising results of this approach on small problems, and more recently, also on reinforcement learning problems, the approach has received little attention in the supervised few-shot learning setting. We revisit this approach and test it on modern few-shot learning benchmarks. We find that LSTM, surprisingly, outperform the popular meta-learning technique MAML on a simple few-shot sine wave regression benchmark, but that LSTM, expectedly, fall short on more complex few-shot image classification benchmarks. We identify two potential causes and propose a new method called Outer Product LSTM (OP-LSTM) that resolves these issues and displays substantial performance gains over the plain LSTM. Compared to popular meta-learning baselines, OP-LSTM yields competitive performance on within-domain few-shot image classification, and performs better in cross-domain settings by 0.5–1.9% in accuracy score. While these results alone do not set a new state-of-the-art, the advances of OP-LSTM are orthogonal to other advances in the field of meta-learning, yield new insights in how LSTM work in image classification, allowing for a whole range of new research directions. For reproducibility purposes, we publish all our research code publicly.

show abstract

Subspace Adaptation Prior for Few-Shot Learning

Cited by 2 publications

References 38 publications

Cross-channel spatial attention network for few-shot classification

Cross-channel spatial attention network for few-shot classification

Are LSTMs good few-shot learners?

Contact Info

Product

Resources

About