Leveraging the Feature Distribution in Transfer-based Few-Shot Learning

Hu, Yuqing; Gripon, Vincent; Pateux, Stéphane

doi:10.48550/arxiv.2006.03806

Cited by 18 publications

(52 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adopting few-shot image classification as our benchmark [9], we train a neural vision model using a set of background images BI = {(I i , y i )} m i=1 (base set) from K classes with y i ∈ C B = {c 1 , c 2 , . .…”

Section: Proposed Method: Viocementioning

confidence: 99%

Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

Jayathilaka,

Mu,

Sattler

2021

Preprint

View full text Add to dashboard Cite

We propose a novel framework named ViOCE that integrates ontology-based background knowledge in the form of n-ball concept embeddings into a neural network based vision architecture. The approach consists of two components -converting symbolic knowledge of an ontology into continuous space by learning n-ball embeddings that capture properties of subsumption and disjointness, and guiding the training and inference of a vision model using the learnt embeddings. We evaluate ViOCE using the task of few-shot image classification, where it demonstrates superior performance on two standard benchmarks.

show abstract

Section: Proposed Method: Viocementioning

confidence: 99%

Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

Jayathilaka,

Mu,

Sattler

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Pseudo-labeling based methods and graph-based methods are two main lines of efforts. Pseudo-labeling based methods consist of prototype refinement [13,18,19,26], self-training [17,39] and entropy minimization [2,7]. As for graph-based approaches [14,20,28,41], graph models are constructed to propagate information from labeled data to unlabeled data.…”

Section: Semi-supervised Few-shot Learningmentioning

confidence: 99%

“…Upon Meta-Inc-Baseline, we further generalize to S 2 I-FSL in Section 4.2 and Section 4.3. Since prototype refinement using pseudo labels has been proved effective in semi-supervised methods [13,18,19,26], an intuitive solution to extend Meta-Inc-Baseline to S 2 I-FSL is to refine the novel class weights W n with unlabeled data. Meanwhile, base class weights W b trained on abundant base class samples remain unchanged.…”

Section: Prototype Refinement With Fake Unlabeled Datamentioning

confidence: 99%

A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning

Zhao¹,

Guo²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Few-shot learning (FSL) aims to learn models that generalize to novel classes with limited training samples. Recent works advance FSL towards a scenario where unlabeled examples are also available and propose semi-supervised FSL methods. Another line of methods also cares about the performance of base classes in addition to the novel ones and thus establishes the incremental FSL scenario. In this paper, we generalize the above two under a more realistic yet complex setting, named by Semi-Supervised Incremental Few-Shot Learning (S 2 I-FSL). To tackle the task, we propose a novel paradigm containing two parts: (1) a well-designed meta-training algorithm for mitigating ambiguity between base and novel classes caused by unreliable pseudo labels and (2) a model adaptation mechanism to learn discriminative features for novel classes while preserving base knowledge using few labeled and all the unlabeled data. Extensive experiments on standard FSL, semi-supervised FSL, incremental FSL, and the firstly built S 2 I-FSL benchmarks demonstrate the effectiveness of our proposed method.

show abstract

“…Bateni et al's proposed Simple CNAPS 14 follows a similar metric-based clustering, however, a Mahalanobis distance is used for comparison between points, rather than propagation of labels. PT+MAP 17 and LaplacianShot 18 function similarly, however, both propose alternative strategies for distance metrics when considering query and support points. AmdimNet 19 and S2M2 20 , alternatively, leverage self-supervised techniques in order to generate a stronger embedding-space mapping for input data.…”

Section: Transductive and Self-supervised Approaches To Few-shot Lear...mentioning

confidence: 99%

“…Technique Backbone Preprocessing Extra Training Data AmdimNet 19 Self-supervised Metric AmdimNet No Yes EPNet 16 Transductive Metric WRN28-10 No Yes SimpleCNAPS 14 Metric ResNet18 No Yes PT+MAP 17 Metric WRN28-10 Yes No LaplacianShot 18 Metric WRN28-10 No No S2M2R 20 Self-supervised Metric WRN28-10 Yes No Reptile 13 Optimization CONV4 No No MAML 12 Optimization CONV4 No No ProtoNet 15 Metric CONV4 No No Table 1. An overview of the differing details between the models trained and tested.…”

Section: Model Evaluation Table Model Namementioning

confidence: 99%

Automated Human Cell Classification in Sparse Datasets using Few-Shot Learning

Walsh,

Abdelpakey,

Shehata

et al. 2021

Preprint

View full text Add to dashboard Cite

Classifying and analyzing human cells is a lengthy procedure, often involving a trained professional. In an attempt to expedite this process, an active area of research involves automating cell classification through use of deep learning-based techniques. In practice, a large amount of data is required to accurately train these deep learning models. However, due to the sparse human cell datasets currently available, the performance of these models is typically low. This study investigates the feasibility of using few-shot learning-based techniques to mitigate the data requirements for accurate training. The study is comprised of three parts: First, current state-of-the-art few-shot learning techniques are evaluated on human cell classification. The selected techniques are trained on a non-medical dataset and then tested on two out-of-domain, human cell datasets. The results indicate that, overall, the test accuracy of state-of-the-art techniques decreased by at least 30% when transitioning from a non-medical dataset to a medical dataset. Reptile and EPNet were the top performing techniques tested on the BCCD dataset and HEp-2 dataset respectively. Second, this study evaluates the potential benefits, if any, to varying the backbone architecture and training schemes in current state-of-the-art few-shot learning techniques when used in human cell classification. To this end, the best technique identified in the first part of this study, EPNet, is used for experimentation. In particular, the study used 6 different network backbones, 5 data augmentation methodologies, and 2 model training schemes. Even with these additions, the overall test accuracy of EPNet decreased from 88.66% on non-medical datasets to 44.13% at best on the medical datasets. Third, this study presents future directions for using few-shot learning in human cell classification. In general, few-shot learning in its current state performs poorly on human cell classification. The study proves that attempts to modify existing network architectures are not effective and concludes that future research effort should be focused on improving robustness towards out-of-domain testing using optimization-based or self-supervised few-shot learning techniques.

show abstract

Leveraging the Feature Distribution in Transfer-based Few-Shot Learning

Cited by 18 publications

References 29 publications

Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification

A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning

Automated Human Cell Classification in Sparse Datasets using Few-Shot Learning

Contact Info

Product

Resources

About