Tensor feature hallucination for few-shot learning

Lazarou, Michalis; Stathaki, Tania; Avrithis, Yannis

doi:10.1109/wacv51458.2022.00211

Cited by 19 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Few-Shot Learning and Generation. Most few-shot learning methods fall into three categories: meta-learning (Finn, Abbeel, and Levine 2017;Oreshkin, López, and Lacoste 2018;Vinyals et al 2016), transfer-learning (Yang, Wang, and Zhu 2022;Zhang et al 2022b;Hu et al 2022), and feature augmentations (Lazarou, Stathaki, and Avrithis 2022;Chen et al 2018;Ye et al 2020). These methods use textual descriptions of novel classes to generate and align images, promoting the effective use of synthetic images in training few-shot learners.…”

Section: Related Workmentioning

confidence: 99%

HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback

Han,

Huang,

Gong

et al. 2024

AAAI

View full text Add to dashboard Cite

We introduce HuTuMotion, an innovative approach for generating natural human motions that navigates latent motion diffusion models by leveraging few-shot human feedback. Unlike existing approaches that sample latent variables from a standard normal prior distribution, our method adapts the prior distribution to better suit the characteristics of the data, as indicated by human feedback, thus enhancing the quality of motion generation. Furthermore, our findings reveal that utilizing few-shot feedback can yield performance levels on par with those attained through extensive human feedback. This discovery emphasizes the potential and efficiency of incorporating few-shot human-guided optimization within latent diffusion models for personalized and style-aware human motion generation applications. The experimental results show the significantly superior performance of our method over existing state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback

Han,

Huang,

Gong

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…The intrinsic signal brings together clusters of hallucinated and real examples that belong to the same class while pushing apart clusters from different classes. TFH‐ft [31] argues that most hallucination methods focus on generating feature vectors that are commonly obtained by global average pooling on the output feature maps. As a result, spatial details that might be necessary for modelling the underlying data distribution are discarded.…”

Section: Related Workmentioning

confidence: 99%

Task‐oriented feature hallucination for few‐shot image classification

Gao

Xiao-peng

2023

IET Image Processing

View full text Add to dashboard Cite

Data hallucination generates additional training examples for novel classes to alleviate the data scarcity problem in few‐shot learning (FSL). Existing hallucination‐based FSL methods normally train a general embedding model first by applying information extracted from base classes that have abundant data. In those methods, hallucinators are then built upon the trained embedding model to generate data for novel classes. However, these hallucination methods usually rely on general‐purpose embeddings, limiting their ability to generate task‐oriented samples for novel classes. Recent studies have shown that task‐specific embedding models, which are adapted to novel tasks, can achieve better classification performance. To improve the performance of example hallucination for tasks, a task‐oriented embedding model is used in the proposed method to perform task‐oriented generation. After the initialization, the hallucinator is finetuned by applying a task‐oriented embedding model with the guidance of a teacher–student mechanism. The proposed task‐oriented hallucination method contains two steps. An initial embedding network and an initial hallucinator are trained with a base dataset in the first step. The second step contains a pseudo‐labelling process where the base dataset is pseudo‐labelled using support data of the few‐shot task and a task‐oriented fine‐tuning process where the embedding network and hallucinator are adjusted simultaneously. Both the embedding network and the hallucinator are updated with the support set and the pseudo‐labelled base dataset using knowledge distillation. The experiments are conducted on four popular few‐shot datasets. The results demonstrate that the proposed approach outperforms state‐of‐the‐art methods with 0.8% to 4.08% increases in classification accuracy for 5‐way 5‐shot tasks. It also achieves comparable accuracy to state‐of‐the‐art methods for 5‐way 1‐shot tasks.

show abstract

“…Hallucination Methods. Feature Hallucination of examples is first introduced for visual recognition (Hariharan and Girshick, 2017) by meta-learning (Wang et al, 2018), variational inference (Luo et al, 2021;Lazarou et al, 2022), and adversarial learning Tjio et al, 2022). Label Hallucination (Jian and Torresani, 2022) assigns soft pseudo-labels for unlabelled images to extend the fine-tuning few-shot dataset.…”

Section: Related Workmentioning

confidence: 99%

Embedding Hallucination for Few-Shot Language Fine-tuning

Jian¹,

Gao²,

Vosoughi³

2022

Preprint

View full text Add to dashboard Cite

Few-shot language learners adapt knowledge from a pre-trained model to recognize novel classes from a few-labeled sentences. In such settings, fine-tuning a pre-trained language model can cause severe over-fitting. In this paper, we propose an Embedding Hallucination (EmbedHalluc) method, which generates auxiliary embedding-label pairs to expand the finetuning dataset. The hallucinator is trained by playing an adversarial game with the discriminator, such that the hallucinated embedding is indiscriminative to the real ones in the finetuning dataset. By training with the extended dataset, the language learner effectively learns from the diverse hallucinated embeddings to overcome the over-fitting issue. Experiments demonstrate that our proposed method is effective in a wide range of language tasks, outperforming current fine-tuning methods. Further, we show that EmbedHalluc outperforms other methods that address this over-fitting problem, such as common data augmentation, semi-supervised pseudo-labeling, and regularization. The code will be made available at: https://github.com/yiren-jian/EmbedHalluc.

show abstract

Tensor feature hallucination for few-shot learning

Cited by 19 publications

References 27 publications

HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback

HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback

Task‐oriented feature hallucination for few‐shot image classification

Embedding Hallucination for Few-Shot Language Fine-tuning

Contact Info

Product

Resources

About