Learning to teach and learn for semi-supervised few-shot image classification

Li, Xinzhe; Huang, Jianqiang; Liu, Yaoyao; Zhou, Qin; Zheng, Shuang; Schiele, Bernt; Sun, Quansen

doi:10.1016/j.cviu.2021.103270

Cited by 71 publications

(107 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some methods transform the existing training set data and assign pseudo-labels to add new data to the training set [7,86,145]. Another line of strategies [45,123,276] give high-confidence pseudo-labels to unlabeled data sets to turn them into training data. Also, the similar data sets can be used as the source to generate data into the few-shot training set [70,216].…”

Section: Few-shot Learningmentioning

confidence: 99%

What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data

Hu¹,

Chapman²,

Wen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Supervised machine learning has several drawbacks that make it difficult to use in many situations. Drawbacks include: heavy reliance on massive training data, limited generalizability and poor expressiveness of high-level semantics. Low-shot Learning attempts to address these drawbacks. Low-shot learning allows the model to obtain good predictive power with very little or no training data, where structured knowledge plays a key role as a high-level semantic representation of human. This article will review the fundamental factors of low-shot learning technologies, with a focus on the operation of structured knowledge under different low-shot conditions. We also introduce other techniques relevant to low-shot learning. Finally, we point out the limitations of low-shot learning, the prospects and gaps of industrial applications, and future research directions.

show abstract

Section: Few-shot Learningmentioning

confidence: 99%

What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data

Hu¹,

Chapman²,

Wen³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, a number of methods have been proposed to leverage unlabeled data beyond the N examples from the C classes to enhance model accuracy. They either use the unlabeled data in the training (semi-supervised learning) or perform classifications with a set of query data together (transductive learning) [17,12,35,9,27,18]. In this work, we focus on the traditional FSL setting where no additional information or unlabeled data are available and the prediction for a query data point is made independently from (without knowing) any other query data points.…”

Section: Few-shot Learning Problemsmentioning

confidence: 99%

“…Each individual model in our ensemble employs a relation network for classification while the relational network in different models receives different representation as input. Many recent FSL studies proposed approaches that utilized extra unlabled data or other additional information [17,12,35,9,27,18]. They are not in the scope of the problem we were considering and thus not compared in the result section.…”

Section: Related Workmentioning

confidence: 99%

Multi-Level Contrastive Learning for Few-Shot Problems

Chen¹,

Zhang²

2021

Preprint

View full text Add to dashboard Cite

Contrastive learning is a discriminative approach that aims at grouping similar samples closer and diverse samples far from each other. It it an efficient technique to train an encoder generating distinguishable and informative representations, and it may even increase the encoder's transferability. Most current applications of contrastive learning benefit only a single representation from the last layer of an encoder.In this paper, we propose a multi-level contrasitive learning approach which applies contrastive losses at different layers of an encoder to learn multiple representations from the encoder. Afterward, an ensemble can be constructed to take advantage of the multiple representations for the downstream tasks. We evaluated the proposed method on few-shot learning problems and conducted experiments using the mini-ImageNet and the tiered-ImageNet datasets. Our model achieved the new state-of-the-art results for both datasets, comparing to previous regular, ensemble, and contrastive learing (single-level) based approaches.Preprint. Under review.

show abstract

“…Tasks We consider N -way, K-shot classification tasks with N = 5 randomly sampled novel classes and K ∈ {1, 5} examples drawn at random per class as support set S, that is, L = 5K examples in total. For the query set Q, we draw 15 additional examples per class, that is, 75 examples in total, which is the most common choice [39,35,68].…”

Section: Setupmentioning

confidence: 99%

Tensor feature hallucination for few-shot learning

Lazarou¹,

Stathaki²,

Avrithis³

2021

Preprint

View full text Add to dashboard Cite

Few-shot classification addresses the challenge of classifying examples given not just limited supervision but limited data as well. An attractive solution is synthetic data generation. However, most such methods are overly sophisticated, focusing on high-quality, realistic data in the input space. It is unclear whether adapting them to the few-shot regime and using them for the downstream task of classification is the right approach. Previous works on synthetic data generation for few-shot classification focus on exploiting complex models, e.g. a Wasserstein GAN with multiple regularizers or a network that transfers latent diversities from known to novel classes.We follow a different approach and investigate how a simple and straightforward synthetic data generation method can be used effectively. We make two contributions, namely we show that: (1) using a simple loss function is more than enough for training a feature generator in the few-shot setting; and (2) learning to generate tensor features instead of vector features is superior. Extensive experiments on miniImagenet, CUB and CIFAR-FS datasets show that our method sets a new state of the art, outperforming more sophisticated few-shot data augmentation methods.

show abstract

Learning to teach and learn for semi-supervised few-shot image classification

Cited by 71 publications

References 15 publications

What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data

What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data

Multi-Level Contrastive Learning for Few-Shot Problems

Tensor feature hallucination for few-shot learning

Contact Info

Product

Resources

About