On Data and Algorithms: Understanding Inductive Performance

Abstract

Abstract. In this paper we address two symmetrical issues, the discovery of similarities among classification algorithms, and among datasets. Both on the basis of error measures, which we use to define the error correlation between two algorithms, and determine the relative performance of a list of algorithms. We use the first to discover similarities between learners, and both of them to discover similarities between datasets. The latter sketch maps on the dataset space. Regions within each map exhibit specific patterns of error correlation or relative performance. To acquire an understanding of the factors determining these regions we describe them using simple characteristics of the datasets. Descriptions of each region are given in terms of the distributions of dataset characteristics within it.

Selecting Machine Learning Algorithms Using the Ranking Meta-Learning Approach

“…One of the major challenges in many domains of Computational Intelligence, Machine Learning, Data Analysis and other fields is to investigate the capabilities and limitations of the existing algorithms in order to identify when one algorithm is more adequate than another to solve particular problems [1]. Traditional approaches to selecting algorithms involve, in general, costly trial-and-error procedures, or require expert knowledge, which is not always easy to acquire [2].…”

Meta-learning to optimize the number of hidden nodes of MLP networks trained by Extreme Learning Machine algorithm

“…Each meta-example consists of: (1) meta-attributes that characterizes a learning problem; and (2) a meta-label indicating the best candidate algorithm for the problem. The meta-attributes are, in general, statistics that describe the dataset, such number of examples, number of attributes, correlation between attributes, average entropy of attributes, among others [18] [19]. The meta-label, in turn, is in general a class label indicating the best algorithm for the problem, usually determined by an empirical evaluation (for instance, by a cross validation experiment).…”

Selecting Neural Network Forecasting Models Using the Zoomed-Ranking Approach

“…In [14,15], for instance, a set of different meta-learners is used not only to predict the best base-learner (as described above) but also to recommend a ranking of base-learners. In this approach, a strict meta-learner is built for each different pair (X, Y) of base-learners.…”