Analysis of the consistency of a mixed integer programming-based multi-category constrained discriminant model

Abstract

Classification is concerned with the development of rules for the allocation of observations to groups, and is a fundamental problem in machine learning. Much of previous work on classification models investigates two-group discrimination. Multi-category classification is less-often considered due to the tendency of generalizations of two-group models to produce misclassification rates that are higher than desirable. Indeed, producing "good" two-group classification rules is a challenging task for some applications, and producing good multi-category rules is generally more difficult. Additionally, even when the "optimal" classification rule is known, inter-group misclassification rates may be higher than tolerable for a given classification model. We investigate properties of a mixed-integer programming based multi-category classification model that allows for the pre-specification of limits on inter-group misclassification rates. The mechanism by which the limits are satisfied is the use of a reserved judgment region, an artificial category into which observations are placed whose attributes do not sufficiently indicate membership to any particular group. The method is shown to be a consistent estimator of a classification rule with misclassification limits, and performance on simulated and real-world data is demonstrated.

Support Vector Machines with the Ramp Loss and the Hard Margin Loss

“…Traditional SVM is tuned for the same parameter values except for the loss function. Classification trees are tuned for the split criterion (Gini or information) and k-nearest neighbor is tuned for k (1,3,4,7,9) and distance function (L 1 and L 2 ). Random forests and logistic regression are used with default settings for all tests.…”

“…Steinwart [36] proves that SVM with the traditional hinge loss is universally consistent. Brooks and Lee [7] prove that an integer-programming based method for constrained discrimination, a generalization of the classification problem, is consistent. This paper presents new integer programming formulations for SVM with the ramp loss and hard margin loss that accommodate the use of nonlinear kernel functions and the quadratic margin term.…”

Practice Characteristics That Influence Nonurgent Pediatric Emergency Department Utilization

“…The DAMIP classification model, a general-purpose, optimization-based, predictive modeling framework, has proven to be a very powerful supervised learning classification approach in predicting various biomedical and clinical phenomena [22][23][24] due to the universal consistency of the resulting classification rules and their ability to classify with high accuracy even among small training sets. 25 Fifty percent of the practices were randomly selected as the training set, 7 practices in group 1 and 9 in group 2. The DAMIP model was then applied to this training set to develop the prediction rule and to obtain an unbiased estimate of correct classification.…”

Supersparse linear integer models for optimized medical scoring systems

“…6 and 7, surrogate functions provide a poor trade-off between accuracy and sparsity. Convex surrogate loss functions, for instance, produce models that do not attain the best learning-theoretic guarantee on predictive accuracy and are not robust to outliers (Li and Lin 2007;Brooks and Lee 2010;Nguyen and Sanner 2013). Similarly, 1 -regularization is only guaranteed to recover the correct sparse solution (i.e., the one that minimizes the 0 -norm) under restrictive conditions that are rarely satisfied in practice (Zhao and Bin 2007;Liu and Zhang 2009).…”