THORS: An Efficient Approach for Making Classifiers Cost-Sensitive

Abstract

In this paper, we propose an effective TH resholding method based on ORder S tatistic, called THORS, to convert an arbitrary scoring-type classifier, which can induce a continuous cumulative distribution function of the score, into a cost-sensitive one. The procedure, uses order statistic to find an optimal threshold for classification, requiring almost no knowledge of classifiers itself. Unlike common data-driven methods, we analytically show that THORS has theoretical guaranteed performance, theoretical bounds for the costs and lower time complexity. Coupled with empirical results on several real-world data sets, we argue that THORS is the preferred cost-sensitive technique.

MinViME/Minimum Viable Model Estimator

“…The user enters the specification of the problem and the costs and benefits of the decision scenario the model will assist with. This structure follows the cost matrix analysis used in the cost sensitive learning literature [1][2][3][4], and elsewhere used to adjust the decision threshold of classifier systems [5] and estimate the ROI of machine learning solutions [6].…”

Minimum Viable Model Estimates for Machine Learning Projects

“…For example, work on designing loss functions for specific applications (See [1], [2]). Alternatively, there is a practice of using cost sensitive learning [3,4,5,6] to permit general techniques learn a solution that is tuned to the specifics of business problem (for example [7]). In general, you can achieve the same outcome by learning a well calibrated probability estimator and adjusting the decision threshold on the basis of the cost matrix [8].…”