Selective generation of training examples in active meta-learning

Abstract

Abstract. Meta-Learning has been successfully applied to acquire knowledge used to support the selection of learning algorithms. Each training example in Meta-Learning (i.e. each meta-example) is related to a learning problem and stores the experience obtained in the empirical evaluation of a set of candidate algorithms when applied to the problem. The generation of a good set of meta-examples can be a costly process depending for instance on the number of available learning problems and the complexity of the candidate algorithms. In this work, we proposed the Active Meta-Learning, in which Active Learning techniques are used to reduce the set of meta-examples by selecting only the most relevant problems for meta-example generation. In an implemented prototype, we evaluated the use of two different Active Learning techniques applied in two different Meta-Learning tasks. The performed experiments revealed a significant gain in the Meta-Learning performance when the active techniques were used to support the meta-example generation.

A Study of Algorithm Selection in Data Mining using Meta-Learning

“…al. [22] took the 'application of metalearning in the field of data mining algorithm selection' one step further by incorporating the idea of active learning in it.…”

Combining Meta-learning and Active Selection of Datasetoids for Algorithm Selection

“…In [10], active learning is proposed to improve the generation of meta-examples. This proposal, termed as Active Meta-learning, is illustrated in Figure 3.…”

“…Labeling is done by evaluating the candidate algorithms on the selected problem and the best algorithm becomes the label of the corresponding meta-example. The The Active Meta-learning was empirically evaluated in [10] by using an uncertainty sampling method originally proposed in [8] for the k-NN algorithm. This method selects unlabeled examples for which the current k-NN learner has high uncertainty in its prediction.…”

Combining Uncertainty Sampling methods for supporting the generation of meta-examples

“…In [24], we presented the first experiments performed to evaluate the viability of Active Meta-Learning. In that work, different active methods based on Uncertainty Sampling were used to select meta-examples for an instancebased meta-learner.…”

“…In a previous work [24], we proposed the Active Meta-Learning in which Active Learning techniques [8] were used to support the generation of metaexamples. Active Learning is a paradigm of Machine Learning which aims to reduce the number of training examples, at same time maintaining (or even improving) the performance of the learning algorithm.…”