Mixture-based Feature Space Learning for Few-shot Image Classification

Afrasiyabi, Arman; Lalonde, Jean‐François; Gagné, Christian

doi:10.1109/iccv48922.2021.00891

Cited by 55 publications

(19 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our method still achieves the new SOTA results on both of these two benchmarks. ResNet-12 47.76 ± 0.77 65.30 ± 0.76 BML [84] ResNet-12 45.00 ± 0.41 63.03 ± 0.41 ALFA+MeTAL [3] ResNet-12 44.54 ± 0.50 58.44 ± 0.42 MixtFSL [1] ResNet-12 41.50 ± 0.67 58.39 ± 0.62 PAL [41] ResNet-12 47.20 ± 0.60 64.00 ± 0.60 TPMN [67] ResNet-12 46.93 ± 0.71 63.26 ± 0.74 MN + MC [79] ResNet 3. Comparison with the state-of-the-art 5-way 1-shot and 5way 5-shot performance with 95% confidence intervals on FC100.…”

Section: Comparisons With State-of-the-art Resultsmentioning

confidence: 99%

“…At the same time, these surrogates need to be learnt at the same time. Supposing the supervised learning objective of the student is L surr , then the parameters θ s of the student and its associated surrogates can be updated with the following equations In between every two transformer sets, a spectral tokens pooling layer is used to down-sample the patch token number by 1 2 for information aggregation.…”

Section: Jointly Attribute Surrogates and Parameters Learningmentioning

confidence: 99%

“…During training, every student transformer builds and updates a surrogate descriptor for each category. The patch tokens integrated through the attention map are also used to generate patch surrogate descriptors in the first stage.In between every two transformer sets, a spectral tokens pooling layer is used to down-sample the patch token number by1 2 for information aggregation.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning

He¹,

Liang²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper presents new hierarchically cascaded transformers that can improve data efficiency through attribute surrogates learning and spectral tokens pooling. Vision transformers have recently been thought of as a promising alternative to convolutional neural networks for visual recognition. But when there is no sufficient data, it gets stuck in overfitting and shows inferior performance. To improve data efficiency, we propose hierarchically cascaded transformers that exploit intrinsic image structures through spectral tokens pooling and optimize the learnable parameters through latent attribute surrogates. The intrinsic image structure is utilized to reduce the ambiguity between foreground content and background noise by spectral tokens pooling. And the attribute surrogate learning scheme is designed to benefit from the rich visual information in image-label pairs instead of simple visual concepts assigned by their labels. Our Hierarchically Cascaded Transformers, called HCTransformers, is built upon a self-supervised learning framework DINO and is tested on several popular few-shot learning benchmarks.In the inductive setting, HCTransformers surpass the DINO baseline by a large margin of 9.7% 5-way 1shot accuracy and 9.17% 5-way 5-shot accuracy on miniImageNet, which demonstrates HCTransformers are efficient to extract discriminative features. Also, HCTransformers show clear advantages over SOTA few-shot classification methods in both 5-way 1-shot and 5-way 5-shot settings on four popular benchmark datasets, including miniImageNet, tieredImageNet, FC100, and CIFAR-FS. The trained weights and codes are available at https: //github.com/StomachCold/HCTransformers.

show abstract

Section: Comparisons With State-of-the-art Resultsmentioning

confidence: 99%

Section: Jointly Attribute Surrogates and Parameters Learningmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning

He¹,

Liang²,

Zhao³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Transfer learning. These methods [5,13,3,12,8,21,1,23,7,2,16,36] take advantage of the standard transfer learning pipeline, which first pretrain a model on base classes, then revise the feature embeddings output by the pretrained model with limited novel samples. [5] used cosine classifier to normalize the magnitude of both embeddings and classification weights for compacting the intra-class intensity.…”

Section: Related Workmentioning

confidence: 99%

“…Naturally, fewshot learning [35] aims at learning from scarce data, which is already studied long before the deep learning era. In this paper, we focus on the image recognition task, also known as few-shot image classification, a widely studied few-shot task [33,29,9,30,5,21,1,23,7,2,36]. Deep learning techniques have further pushed few-shot learning's average accuracy over multiple runs (i.e., episodes) towards a high level that appears to be already applicable to real-world applications.…”

Section: Introductionmentioning

confidence: 99%

Worst Case Matters for Few-Shot Recognition

Fu¹,

Cao²,

Wu³

2022

Preprint

View full text Add to dashboard Cite

Few-shot recognition learns a recognition model with very few (e.g., 1 or 5) images per category, and current few-shot learning methods focus on improving the average accuracy over many episodes. We argue that in real-world applications we may often only try one episode instead of many, and hence maximizing the worst-case accuracy is more important than maximizing the average accuracy. We empirically show that a high average accuracy not necessarily means a high worst-case accuracy. Since this objective is not accessible, we propose to reduce the standard deviation and increase the average accuracy simultaneously. In turn, we devise two strategies from the bias-variance tradeoff perspective to implicitly reach this goal: a simple yet effective stability regularization (SR) loss together with model ensemble to reduce variance during fine-tuning, and an adaptability calibration mechanism to reduce the bias. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed strategies, which outperforms current stateof-the-art methods with a significant margin in terms of not only average, but also worst-case accuracy.

show abstract

Associative Alignment for Few-Shot Image Classification

Afrasiyabi

Lalonde

Gagné

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In image classification, it is common practice to train deep networks to extract a single feature vector per input image. Few-shot classification methods also mostly follow this trend. In this work, we depart from this established direction and instead propose to extract sets of feature vectors for each image. We argue that a set-based representation intrinsically builds a richer representation of images from the base classes, which can subsequently better transfer to the few-shot classes. To do so, we propose to adapt existing feature extractors to instead produce sets of feature vectors from images. Our approach, dubbed SetFeat, embeds shallow self-attention mechanisms inside existing encoder architectures. The attention modules are lightweight, and as such our method results in encoders that have approximately the same number of parameters as their original versions. During training and inference, a set-to-set matching metric is used to perform image classification. The effectiveness of our proposed architecture and metrics is demonstrated via thorough experiments on standard few-shot datasetsnamely miniImageNet, tieredImageNet, and CUB-in both the 1-and 5-shot scenarios. In all cases but one, our method outperforms the state-of-the-art.

show abstract

Mixture-based Feature Space Learning for Few-shot Image Classification

Cited by 55 publications

References 41 publications

Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning

Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning

Worst Case Matters for Few-Shot Recognition

Associative Alignment for Few-Shot Image Classification

Contact Info

Product

Resources

About