A Linear Programming Approach to Discriminant Analysis with a Reserved-Judgment Region

Abstract

A linear-programming model is proposed for deriving discriminant rules that allow allocation of entities to a reserved-judgment region. The size of the reserved-judgment region, which can be controlled by varying parameters within the model, dictates the level of aggressiveness (cautiousness) of allocating (misallocating) entities to groups. Results of simulation experiments for various configurations of normal and contaminated normal three-group populations are reported for a variety of parameter selections. Results of cross-validation experiments using real data sets are also reported. Both the simulation and cross-validation experiments include comparison with other discriminant analysis techniques. The results demonstrate that the proposed model is useful for deriving discriminant rules that reduce the chances of misclassification, while maintaining a reasonable level of correct classification.

Large-Scale Optimization-Based Classification Models in Medicine and Biology

“…Feasible solutions obtained from our classification models correspond to predictive rules. Empirical results 33,56 indicate that the resulting classification model instances are computationally very challenging, and even intractable by competitive commercial MIP solvers. However, the resulting predictive rules prove to be very promising, offering correct classification rates on new unknown data ranging from 80 to 100% on various types of biological/medical problems.…”

“…The results, reported in Gallagher et al, 33 and Lee et al, 56 show the methods are promising, based on applications to both simulated data and real-application datasets from the machine learning database repository. 69 Furthermore, our methods compare well to existing methods, often producing better results when compared to other approaches such as artificial neural networks, quadratic discriminant analysis, tree classification, and other support-vector machines.…”

“…Utilizing the technology of large-scale discrete optimization and support-vector machines, we have developed novel predictive models that simultaneously include the following features: (1) the ability to classify any number of distinct groups; (2) the ability to incorporate heterogeneous types of attributes as input; (3) a high-dimensional data transformation that eliminates noise and errors in biological data; (4) constraints to limit the rate of misclassification, and a reserved-judgment region that provides a safeguard against over-training (which tends to lead to high misclassification rates from the resulting predictive rule); and (5) successive multi-stage classification capability to handle data points placed in the reservedjudgment region. Based on the description in Gallagher et al, 32,33 Lee et al, 56 and Lee, 49,50 we summarize below some of the classification models we have developed.…”

“…Extensive computational experiments have been performed to gauge the qualities of solutions of different models. 14,33,49,50,56 Computational Strategies…”

“…Our model, the first discrete support-vector machine, offers distinct features (e.g., the ability to classify any number of groups, management of the curse of dimensionality in data attributes, and a reserved-judgment region to facilitate multi-stage classification analysis) that are not simultaneously available in existing classification software. 32,33,49,50,56 Studies involving tumor volume identification, ultrasonic cell disruption in drug delivery, lung tumor cell motility analysis, CpG island aberrant methylation in human cancer, predicting early atherosclerosis using biomarkers, and fingerprinting native and angiogenic microvascular networks using functional perfusion data indicate that our zapproach is adaptable and can produce effective and reliable predictive rules for various biomedical and bio-behavior phenomena. 13,25,26,51,53,55,57 , Section ''Background'' briefly describes the background of discriminant analysis.…”