Multicategory discrimination via linear programming

Abstract

A single linear program is proposed for discriminating between the elements of k disjoint point sets in the n-dimensional real space Rn. When the conical hulls of the k sets are (k-1)-point disjoint in R"+', a k-piece piecewise-linear surface generated by the linear program completely sepwates the k sets. This improves on a previous linear programming approach which required that each set be linearly separable from the remaining k-1 sets. When the conical hulls of the k sets are not (k-1)-point d~~sjoint. the proposed linear program generates an error-minimizing piecewise-linear separator for the k sets. For this case it is shown that the null solution is never a unique solver of the linear program and occurs only under the rather rare condition when the mean of each point set equals the mean of the means of the other k-1 sets. This makes the proposed linear computational programming formulation useful for approximately discriminating between k sets that are not piecewise-linear separable. Computational results are reported for three previously available databases.

An Introduction to Boosting and Leveraging

“…We comment that the original bound in [192] contained an extra factor of log N multiplying VCdim(F) in (15).…”

“…In this section we present a brief survey of several extensions and generalizations, although many others exist, e.g. [76,158,62,18,31,3,155,15,44,52,82,164,66,38,137,147,16,80,185,100,14].…”

“…One uses the example weights computed at each iteration to determine which examples are highly influential and hard to classify. Assuming that the hard examples are "noisy examples", the algorithm chooses the mistrust parameter at iteration t, ζ (t) n , as the amount by which the example (x n , y n ) influenced the decision in previous iterations: 15 Note that we use w to mark the parameters of the hyperplane in feature space and α to denote the coefficients generated by the algorithm (see Section 4.1).…”

Untitled

“…Perceptron Decision Trees (PDT) have been introduced by a number of authors under different names (Mangasarian et al, 1990;Bennett & Mangasarian, 1992, 1994a, 1994bBreiman et al, 1984;Broadley & Utgoff, 1995;Utgoff, 1989;Murthy, Kasif, & Salzberg, 1994). They are decision trees in which each internal node is associated with a hyperplane in general position in the input space.…”

“…They are decision trees in which each internal node is associated with a hyperplane in general position in the input space. They have been used in many real-world pattern classification tasks with good results (Bennett & Mangasarian 1994a;Murthy, Kasif, & Salzberg, 1994;Bennett, Wu, & Auslender, 1998). Given their high flexibility, a feature that they share with more standard decision trees such as the ones produced by C4.5 (Quinlan, 1993), they tend to overfit the data if their complexity is not somehow kept under control.…”