Leveraging the Feature Distribution in Transfer-Based Few-Shot Learning

Hu, Yuqing; Gripon, Vincent; Pateux, Stéphane

doi:10.1007/978-3-030-86340-1_39

Cited by 100 publications

(114 citation statements)

References 15 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…SSR [44] 72.40 ± 0.60 80.20 ± 0.40 fine-tuning(train+val) [49] 68.11 ± 0.69 80.36 ± 0.50 SIB+E 3 BM [50] 71.40 81.20 LR+DC [17] 68.57 ± 0.55 82.88 ± 0.42 EPNet [31] 70.74 ± 0.85 84.34 ± 0.53 TIM-GD [42] 77.80 87.40 PT+MAP [51] 82.92 ± 0.26 88.82 ± 0.13 iLPC [45] 83.05 ± 0.79 88.82 ± 0.42 ODC [43] 80.64 ± 0.34 89.39 ± 0.39 PEMnE-BMS * [32] 83. 35…”

Section: Table IV 1-shot and 5-shot Accuracy Of State-of-the-art Meth...mentioning

confidence: 99%

“…PT+MAP [51] 85.67 ± 0.26 90.45 ± 0.14 TIM-GD [42] 79.90 88.50 ODC [43] 83.73 ± 0.36 90.46 ± 0.46 SSR [44] 81.20 ± 0.60 85.70 ± 0.40 Rot+KD+POODLE [48] 79.67 86.96 DPGN [46] 72.45 ± 0.51 87.24 ± 0.39 EPNet [31] 76.53 ± 0.87 87.32 ± 0.64 ECKPN [47] 73.59 ± 0.45 88.13 ± 0.28 iLPC [45] 83.49 ± 0.88 89.48 ± 0.47 ASY ResNet12 (ours)…”

Section: Table IV 1-shot and 5-shot Accuracy Of State-of-the-art Meth...mentioning

confidence: 99%

See 1 more Smart Citation

EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

Bendou¹,

Hu²,

Lafargue³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Few-shot learning aims at leveraging knowledge learned by one or more deep learning models, in order to obtain good classification performance on new problems, where only a few labeled samples per class are available. Recent years have seen a fair number of works in the field, introducing methods with numerous ingredients. A frequent problem, though, is the use of suboptimally trained models to extract knowledge, leading to interrogations on whether proposed approaches bring gains compared to using better initial models without the introduced ingredients. In this work, we propose a simple methodology, that reaches or even beats state of the art performance on multiple standardized benchmarks of the field, while adding almost no hyperparameters or parameters to those used for training the initial deep learning models on the generic dataset. This methodology offers a new baseline on which to propose (and fairly compare) new techniques or adapt existing ones.

show abstract

Section: Table IV 1-shot and 5-shot Accuracy Of State-of-the-art Meth...mentioning

confidence: 99%

Section: Table IV 1-shot and 5-shot Accuracy Of State-of-the-art Meth...mentioning

confidence: 99%

EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

Bendou¹,

Hu²,

Lafargue³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Compositional Feature Transformation. It is believed that FSL algorithms favor features with more Gaussian-like distributions, and thus various kinds of transformations are used to improve the normality of feature distribution, including power transformation (Hu et al, 2021), Tukey's Ladder of Powers Transformation (Yang et al, 2021), and L2 normalization (Wang et al, 2019). While these transformations are normally used independently, here we propose to combine several transformations sequentially in order to enlarge the expressivity of transformation function and to increase the polymorphism of the FSL process.…”

Section: Deepvoro: Integrating Multi-level Heterogeneity Of Fslmentioning

confidence: 99%

Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach

Ma¹,

Huang²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-shot learning (FSL) is the process of rapid generalization from abundant base samples to inadequate novel samples. Despite extensive research in recent years, FSL is still not yet able to generate satisfactory solutions for a wide range of real-world applications. To confront this challenge, we study the FSL problem from a geometric point of view in this paper. One observation is that the widely embraced ProtoNet model is essentially a Voronoi Diagram (VD) in the feature space. We retrofit it by making use of a recent advance in computational geometry called Cluster-induced Voronoi Diagram (CIVD). Starting from the simplest nearest neighbor model, CIVD gradually incorporates cluster-to-point and then cluster-to-cluster relationships for space subdivision, which is used to improve the accuracy and robustness at multiple stages of FSL. Specifically, we use CIVD (1) to integrate parametric and nonparametric few-shot classifiers; (2) to combine feature representation and surrogate representation; (3) and to leverage feature-level, transformation-level, and geometry-level heterogeneities for a better ensemble. Our CIVD-based workflow enables us to achieve new state-of-the-art results on mini-ImageNet, CUB, and tiered-ImagenNet datasets, with ∼2%−5% improvements upon the next best. To summarize, CIVD provides a mathematically elegant and geometrically interpretable framework that compensates for extreme data insufficiency, prevents overfitting, and allows for fast geometric ensemble for thousands of individual VD. These together make FSL stronger.

show abstract

“…(3) We show that the proposed method can bring large increase in accuracy with a variety of feature extractors and datasets, leading to state-of-the-art results in the considered benchmarks. This work is an extended version of [9], with the main difference that here we consider the broader case where we do not know the proportion of samples belonging to each considered class in the case of transductive few-shot, leading to a new algorithm called Boosted Min-size Sinkhorn. We also propose more efficient preprocessing steps, leading to overall better performance in both inductive and transductive settings.…”

Section: Contributions Let Us Highlight the Main Contributions Of Thi...mentioning

confidence: 99%

“…Namely, the model learns a set of initialization parameters that are in an advantageous position for the model to adapt to a new (small) dataset. Recently, the trend evolved towards using well-thought-out transfer architectures (called backbones) [4,5,6,7,8,9] trained one time on the same training data, but seen as a unique large dataset.…”

Section: Introductionmentioning

confidence: 99%

Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning

Hu¹,

Gripon²,

Pateux³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Few-shot classification is a challenging problem due to the uncertainty caused by using few labelled samples. In the past few years, many methods have been proposed with the common aim of transferring knowledge acquired on a previously solved task, what is often achieved by using a pretrained feature extractor. Following this vein, in this paper we propose a novel transfer-based method which aims at processing the feature vectors so that they become closer to Gaussian-like distributions, resulting in increased accuracy. In the case of transductive few-shot learning where unlabelled test samples are available during training, we also introduce an optimal-transport inspired algorithm to boost even further the achieved performance. Using standardized vision benchmarks, we show the ability of the proposed methodology to achieve state-of-the-art accuracy with various datasets, backbone architectures and few-shot settings.

show abstract

Leveraging the Feature Distribution in Transfer-Based Few-Shot Learning

Cited by 100 publications

References 15 publications

EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients

Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach

Squeezing Backbone Feature Distributions to the Max for Efficient Few-Shot Learning

Contact Info

Product

Resources

About