Task Agnostic Meta-Learning for Few-Shot Learning

Jamal, Muhammad Abdullah; Qi, Guo-Jun

doi:10.1109/cvpr.2019.01199

Cited by 357 publications

(126 citation statements)

References 8 publications

(4 reference statements)

Supporting

Mentioning

112

Contrasting

Order By: Relevance

“…The development of the method is of great significance to the optimization-based meta learning methods. Recently, an expanded task-agnostic meta learning algorithm is proposed to enhance the generalizability of meta-learner towards a variety of tasks, which achieves outstanding performance on few-shot classification and reinforcement learning tasks [179].…”

Section: Optimization In Meta Learningmentioning

confidence: 99%

A Survey of Optimization Methods From a Machine Learning Perspective

Sun

Cao

Han

et al. 2020

IEEE Trans. Cybern.

513

228

View full text Add to dashboard Cite

Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization methods in machine learning face more and more challenges. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and machine learning research. In this paper, we first describe the optimization problems in machine learning. Then, we introduce the principles and progresses of commonly used optimization methods. Next, we summarize the applications and developments of optimization methods in some popular machine learning fields. Finally, we explore and give some challenges and open problems for the optimization in machine learning.

show abstract

Section: Optimization In Meta Learningmentioning

confidence: 99%

A Survey of Optimization Methods From a Machine Learning Perspective

Sun

Cao

Han

et al. 2020

IEEE Trans. Cybern.

513

228

View full text Add to dashboard Cite

show abstract

“…Gradient descent based MAML [3] 48.70 ± 1.84% 63.11 ± 0.92% Reptile [33] 49.97 ± 0.32% 65.99 ± 0.58% meta-SGD [20] 50.47 ± 1.87% 64.03 ± 0.94% LEO [6] 61.76 ± 0.15% 77.46 ± 0.12% MTL [7] 61.20 ± 1.80% 75.50 ± 0.80% TAML [21] 51.73 ± 1.88% 66.05 ± 0.85%…”

Section: Metric Basedmentioning

confidence: 99%

“…Based on MAML, Probabilistic model-agnostic meta-learning [17] introduces probabilistic into model to represent initial data, Kim [18] and Gupta [19] proposed Bayesian way into MAML to establish relationship between post result and pre data by Bayesian prior probability. Meta-SGD [20] and Task-Agnostic Meta-Learning(TAML) [21] try to make meta-learner learn how to learn better rather than only remember initial knowledge.…”

Section: Introductionmentioning

confidence: 99%

Multi-Stage Meta-Learning for Few-Shot with Lie Group Network Constraint

Dong¹,

Li²

2020

Preprint

View full text Add to dashboard Cite

Deep learning has achieved lots of successes in many fields, but when trainable sample are extremely limited, deep learning often under or overfitting to few samples. Meta-learning was proposed to solve difficulties in few-shot learning and fast adaptive areas. Meta-learner learns to remember some common knowledge by training on large scale tasks sampled from a certain data distribution to equip generalization when facing unseen new tasks. Due to the limitation of samples, most approaches only use shallow neural network to avoid overfitting and reduce the difficulty of training process, that causes the waste of many extra information when adapting to unseen tasks. Euclidean space-based gradient descent also make meta-learner's update inaccurate. These issues cause many meta-learning model hard to extract feature from samples and update network parameters. In this paper, we propose a novel method by using multi-stage joint training approach to post the bottleneck during adapting process. To accelerate adapt procedure, we also constraint network to Stiefel manifold, thus meta-learner could perform more stable gradient descent in limited steps. Experiment on mini-ImageNet shows that our method reaches better accuracy under 5-way 1-shot and 5-way 5-shot conditions.

show abstract

“…These systems can offer strong generalization to a corresponding target set. A successful exploitation of the above k-shot learning cases is provided by meta-learning techniques which can be used to deliver effective solutions [6].In this work, we propose a new classification model, which is based on zero-shot philosophy, named MAME-ZsL. The significant advantages of the proposed algorithm is that it reduces computational cost and training time; it avoids potential overfitting by enhancing the learning of features which do not cause issues of exploding or diminishing gradients; and it offers an improved training stability, high generalization performance and remarkable classification accuracy.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification

Demertzis

Iliadis

2020

Algorithms

View full text Add to dashboard Cite

Deep learning architectures are the most effective methods for analyzing and classifying Ultra-Spectral Images (USI). However, effective training of a Deep Learning (DL) gradient classifier aiming to achieve high classification accuracy, is extremely costly and time-consuming. It requires huge datasets with hundreds or thousands of labeled specimens from expert scientists. This research exploits the MAML++ algorithm in order to introduce the Model-Agnostic Meta-Ensemble Zero-shot Learning (MAME-ZsL) approach. The MAME-ZsL overcomes the above difficulties, and it can be used as a powerful model to perform Hyperspectral Image Analysis (HIA). It is a novel optimization-based Meta-Ensemble Learning architecture, following a Zero-shot Learning (ZsL) prototype. To the best of our knowledge it is introduced to the literature for the first time. It facilitates learning of specialized techniques for the extraction of user-mediated representations, in complex Deep Learning architectures. Moreover, it leverages the use of first and second-order derivatives as pre-training methods. It enhances learning of features which do not cause issues of exploding or diminishing gradients; thus, it avoids potential overfitting. Moreover, it significantly reduces computational cost and training time, and it offers an improved training stability, high generalization performance and remarkable classification accuracy. Algorithms 2020, 13, 61 2 of 25 Algorithms 2020, 13, 61 3 of 25In contrast to the learning process of the traditional neural networks, it is not sufficient for the network to learn good representations of the training classes, as the testing classes are distinct and they are not presented in training. However, it is desirable to learn features which distinguish the existing classes.The evaluation process consists of two distinct stages of the following format [4]:Step 1: Given k examples (value of k-shot), if k = 1, then the process is called one-shot; if k = 5, five-shot, and so on. The parameter k represents the number of labeled samples given to the algorithm by each class. By considering these samples, which comprise the support set, the network is required to classify and eventually adapt to existing classes.Step 2: Unknown examples of the labeled classes are presented randomly, unlike the ones presented in the previous step, which the network is called to correctly classify. The set of examples in this stage is known as the query set.The above procedure (steps) is repeated many times using random classes and examples which are sampled from the testing-evaluation set.As it is immediately apparent from the description of the evaluation process, as the number of classes increases, the task becomes more difficult, because the network has to decide between several alternatives. This means that Zero-shot Learning [5] is clearly more difficult than the one-shot, which is more difficult than the five-shot, and so on. Although humans have the ability to cope with this process, traditional ANN require many more examples to genera...

show abstract

Task Agnostic Meta-Learning for Few-Shot Learning

Cited by 357 publications

References 8 publications

A Survey of Optimization Methods From a Machine Learning Perspective

A Survey of Optimization Methods From a Machine Learning Perspective

Multi-Stage Meta-Learning for Few-Shot with Lie Group Network Constraint

GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification

Contact Info

Product

Resources

About