Sharing to learn and learning to share -- Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta review

Abstract

Integrating knowledge across different domains is an essential feature of human learning. Learning paradigms like transfer learning, meta learning, and multi-task learning reflect the human learning process by exploiting the prior knowledge for new tasks, encouraging faster learning and good generalization for new tasks. This article gives a detailed view of these learning paradigms along with a comparative analysis. The weakness of a learning algorithm turns out to be the strength of another, and thereby merging them is a prevalent trait in the literature. This work delivers a literature review of the articles, which fuses two algorithms to accomplish multiple tasks. A global generic learning network, an ensemble of meta learning, transfer learning, and multi-task learning, is also introduced here, along with some open research questions and directions for future research.

Uso de Meta-Learning em Tarefas de Aprendizado Profundo

“…O processo de meta-learning utiliza um conjunto de tarefas, extraído de um grupo maior de tarefas semelhantes entre si, e consiste em duas fases principais [Upadhyay et al 2021]:…”

“…Uma forma de conseguir classificá-las é fazendo uma distinção sobre a metodologia de aprendizado que pode ser dividida em três tipos: abordagens baseadas em métricas, modelos e otimização. A Figura 3.1 mostra o desempenho de algumas dessas técnicas para a tarefa de classificação de imagens para o conjunto de dados miniImageNet [Vinyals et al 2016].…”

Speeding Up Transformer Training By Using Dataset Subsampling - An Exploratory Analysis

“…Many application domains were studied in previous work, ranging from surveys covering multiple domains (Ruder, 2017;Zhang & Yang, 2017;Thung & Wee, 2018;Vafaeikia et al, 2020;Crawshaw, 2020;Upadhyay et al, 2021;Abhadiomhen et al, 2022), to those dedicated to a specific domain, such as computer vision (Vandenhende et al, 2021) or natural language processing (Zhou, 2019;Worsham & Kalita, 2020;Samant et al, 2022;Zhang et al, 2023). Both traditional ML and deep learning computational models were studied.…”