Meta-Learning Initialization vs Optimizer: Beyond 20 Ways Few-Shot Learning

Aimen, Aroof; Sidheekh, Sahil; Ahuja, Hansin; Krishnan, Narayanan C.

doi:10.2139/ssrn.4266883

Cited by 1 publication

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The differentiation between quick learning and feature reuse is key to attain accurate ANIL characterization [25] (Figure 2). The outer loop's meta-initialization creates a parameter setting that supports quick learning, thus making it possible for the inner loop to rapidly adapt to novel tasks.…”

Section: Maml Using Head Of the Inner Loop (Anil)mentioning

confidence: 99%

LaANIL: ANIL with Look-Ahead Meta-Optimization and Data Parallelism

Tammisetti,

Bierzynski,

Stettinger

et al. 2024

Electronics

View full text Add to dashboard Cite

Meta-few-shot learning algorithms, such as Model-Agnostic Meta-Learning (MAML) and Almost No Inner Loop (ANIL), enable machines to learn complex tasks quickly with limited data and based on previous experience. By maintaining the inner loop head of the neural network, ANIL leads to simpler computations and reduces the complexity of MAML. Despite its benefits, ANIL suffers from issues like accuracy variance, slow initial learning, and overfitting, hardening its adaptation and generalization. This work proposes “Look-Ahead ANIL” (LaANIL), an enhancement to ANIL for better learning. LaANIL reorganizes ANIL’s internal architecture, integrating parallel computing techniques (to process multiple training examples simultaneously across computing units) and incorporating Nesterov momentum (which accelerates convergence by adjusting the learning rate based on past gradient information and extracting informative features for look-ahead gradient computation). These additional features make our model more state-of-the-art capable and better edge-compatible and thus improve few-short learning by enabling models to quickly adapt to new information and tasks. LaANIL’s effectiveness is validated on established meta-few-shot learning datasets, including FC100, CIFAR-FS, Mini-ImageNet, CUBirds-200-2011, and Tiered-ImageNet. The proposed model achieved an increased validation accuracy by 7 ± 0.7% and a variance reduction by 44 ± 4% in two-way two-shot classification as well as increased validation by 5 ± 0.4% and a variance reduction by 18 ± 2% in five-way five-shot classification on the FC100 dataset and similarly performed well on other datasets.

show abstract

Section: Maml Using Head Of the Inner Loop (Anil)mentioning

confidence: 99%