Cost-sensitive meta-learning framework

Abstract

Purpose This paper aims to describe the use of a meta-learning framework for recommending cost-sensitive classification methods with the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?” Design/methodology/approach This paper describes the use of a meta-learning framework for recommending cost-sensitive classification methods for the aim of answering an important question that arises in machine learning, namely, “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?” The framework is based on the idea of applying machine learning techniques to discover knowledge about the performance of different machine learning algorithms. It includes components that repeatedly apply different classification methods on data sets and measures their performance. The characteristics of the data sets, combined with the algorithms and the performance provide the training examples. A decision tree algorithm is applied to the training examples to induce the knowledge, which can then be used to recommend algorithms for new data sets. The paper makes a contribution to both meta-learning and cost-sensitive machine learning approaches. Those both fields are not new, however, building a recommender that recommends the optimal case-sensitive approach for a given data problem is the contribution. The proposed solution is implemented in WEKA and evaluated by applying it on different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system. The developed solution takes the misclassification cost into consideration during the learning process, which is not available in the compared project. Findings The proposed solution is implemented in WEKA and evaluated by applying it to different data sets and comparing the results with existing studies available in the literature. The results show that a developed meta-learning solution produces better results than METAL, a well-known meta-learning system. Originality/value The paper presents a major piece of new information in writing for the first time. Meta-learning work has been done before but this paper presents a new meta-learning framework that is costs sensitive.

Stacked Generalizations in Imbalanced Fraud Data Sets using Resampling Methods

“…The cost of misclassification on the minority class is weighted or penalized more heavily than a correct classification. The true costs include the costs of misclassification, testing and human intervention (Turney, 2002;Shilbayeh et al, 2015), as represented in the Cost Matrix (Table 2) initially developed by Hand (Hand, Whitrow, Adams, Juszczak and Weston, 2008;Bahnsen, Villegas, Aouada, Ottersten, Correa and Villegas, 2017). The administrative costs of a false positive (C FP i = C admin ) are equal to the true positive (C TP i = C admin ), since in the latter instance the account holder needs to be contacted.…”

“…There are two techniques of CSL that can be applied through machine learning. The direct method alters accuracybased algorithms in the way that includes cost weighting for misclassification (Lomax and Vadera, 2013;Shilbayeh et al, 2015). Since the algorithm is developed in a way that rewards lower costs, raising the weight of misclassification will over correct and, therefore, correctly classify.…”

“…Weights of misclassification that are higher for the false positives than false negatives vary inversely as the class size increases. The indirect method applies a wrapper to an algorithm effectively altering how the misclassification is weighted (Shilbayeh et al, 2015). One primary issue is that the initial weighting cost is unknown requiring that a starting point be determined.…”

“…For example, a direct method cost sensitive tree decides how to build branches being controlled by entropy introduced by an information cost factor in selecting how to add attributes (Shilbayeh et al, 2015). Information theory, usually referenced as entropy, quantifies irregularity and measures class distribution consistency (Shannon, 1948;Quinlan, 1987;Lomax et al, 2013).…”

Guest editorial: Artificial intelligence and machine learning in business and management

“…In Shilbayeh and Vadera (2021), the authors report on the use of a meta-learning framework for recommending cost-sensitive classification methods with the aim of answering an important question that arises in machine learning, namely “Among all the available classification algorithms, and in considering a specific type of data and cost, which is the best algorithm for my problem?” The framework is based on the idea of applying ML to discover knowledge about the performance of different ML algorithms. It includes components that repeatedly apply different classification methods on data sets and measure their performance.…”