Meta-learning with an Adaptive Task Scheduler

Abstract

To benefit the learning of a new task, meta-learning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning algorithms randomly sample meta-training tasks with a uniform probability, under the assumption that tasks are of equal importance. However, it is likely that tasks are detrimental with noise or imbalanced given a limited number of meta-training tasks. To prevent the meta-model from being corrupted by such detrimental tasks or dominated by tasks in the majority, in this paper, we propose an adaptive task scheduler (ATS) for the meta-training process. In ATS, for the first time, we design a neural scheduler to decide which meta-training tasks to use next by predicting the probability being sampled for each candidate task, and train the scheduler to optimize the generalization capacity of the metamodel to unseen tasks. We identify two meta-model-related factors as the input of the neural scheduler, which characterize the difficulty of a candidate task to the meta-model. Theoretically, we show that a scheduler taking the two factors into account improves the meta-training loss and also the optimization landscape. Under the setting of meta-learning with noise and limited budgets, ATS improves the performance on both miniImageNet and a real-world drug discovery benchmark by up to 13% and 18%, respectively, compared to state-of-the-art task schedulers.Recently, a few studies have started to consider introducing a task scheduler into meta-learning, by adjusting the class selection strategy for construction of each few-shot classification task [17,19],

Hardness-guided domain adaptation to recognise biomedical named entities under low-resource scenarios

“…However, different domains can have different level of hardness which both works have not yet addressed. Existing domain adaptation techniques using meta learning framework to incorporate hardness information via 1) actively ranking the tasks in term of difficulty level (Yao et al, 2021;Zhou et al, 2020;Liu et al, 2020;Achille et al, 2019); 2) designing an adaptive task scheduler (Yao et al, 2021); or 3) relying on generative approaches to quantify the uncertainties of tasks (Kaddour et al, 2020;Nguyen et al, 2021). To our knowledge, we are the first to perform hardness guided domain adaptation for bioNER tasks.…”