Leveraging Task Variability in Meta-learning

Aimen, Aroof; Ladrecha, Bharat; Sidheekh, Sahil; Krishnan, Narayanan C.

doi:10.1007/s42979-023-01951-6

Cited by 1 publication

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second set of approaches leverages training to select the samples. One possibility is to train a sample retriever, such as a separate scoring model (Rubin et al, 2022;Luo et al, 2023;Wang et al, 2023;Aimen et al, 2023) or using determinantal point process (Yang et al, 2023;Ye et al, 2023a). Another set of approaches leverages reinforcement learning to select good samples Shum et al, 2023;Scarlatos & Lan, 2023).…”

Section: Related Workmentioning

confidence: 99%

FireAnt: Claim-Based Medical Misinformation Detection and Monitoring

Pecher

Srba

Móro

et al. 2021

Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track

View full text Add to dashboard Cite

In few-shot learning, such as meta-learning, fewshot fine-tuning or in-context learning, the limited number of samples used to train a model have a significant impact on the overall success. Although a large number of sample selection strategies exist, their impact on the performance of few-shot learning is not extensively known, as most of them have been so far evaluated in typical supervised settings only. In this paper, we thoroughly investigate the impact of 20 sample selection strategies on the performance of 5 fewshot learning approaches over 8 image and 6 text datasets. In addition, we propose a new method for automatic combination of sample selection strategies (ACSESS) that leverages the strengths and complementary information of the individual strategies. The experimental results show that our method consistently outperforms the individual selection strategies, as well as the recently proposed method for selecting support examples for in-context learning. We also show a strong modality, dataset and approach dependence for the majority of strategies as well as their dependence on the number of shots -demonstrating that the sample selection strategies play a significant role for lower number of shots, but regresses to random selection at higher number of shots.

show abstract

Section: Related Workmentioning

confidence: 99%