Dendogram-based SVM for Multi-Class Classification

Abstract

This paper presents a new approach called dendogrambased support vector machines (DSVM), to treat multiclass problems. First, the method consists to build a taxonomy of classes in an ascendant manner done by ascendant hierarchical clustering method (AHC). Second, SVM is injected at each internal node of the taxonomy in order to separate the two subsets of the current node. Finally, for classifying a pattern query, we present it to the "root" SVM, and then, according to the output, the pattern is presented to one of the two SVMs of the subsets, and so on through the "leaf" nodes. Therefore, the classification procedure is done in a descendant way in the taxonomy from the root through the end level which represents the classes. The pattern is thus associated to one of the last SVMs associated class. AHC decomposition uses distance measures to investigate the class grouping in binary form at each level in the hierarchy. SVM method requires little tuning and yields both high accuracy levels and good generalization for binary classification. Therefore, DSVM method gives good results for multi-class problems by both, training an optimal number of SVMs and by rapidly classifying patterns in a descendant way by selecting an optimal set of SVMs which participate to the final decision. The proposed method is compared to other multi-class SVM methods over several complex problems.

Soft Margin Trees

“…It is expected that the other approaches perform significantly better. -DSVM [19]. Dendrogram-based Support Vector Machines computes the distance matrix by means of the Euclidean distance between the centroids of each pair of classes.…”

“…In [19] the authors present the so called Dendrogram-based Support Vector Machines (DSVM). In order to build the dendrogram, DSVM computes the centroid of each class, and then uses an agglomerative hierarchical algorithm.…”

“…Lately, some authors [7,[18][19][20] have proposed decomposition algorithms which follow a similar approach: learn a decision tree (a special case of DAG) based on a hierarchy of classes. First, these methods build a dendrogram of classes; and then, a binary SVM is learned for each internal node of that hierarchy in order to separate the examples of each subset of classes (see Figures 1 and 2).…”

Learning data structure from classes: A case study applied to population genetics

“…In recent years, some authors [5,26,30,32] have proposed decomposition algorithms to solve multi-class tasks. The underlying idea in all these approaches is the same: to learn a binary decision tree classifier based on a hierarchy of classes.…”

“…All these methods differ basically in the way they build the dendrogram of classes. For instance, Dendrogram-based Support Vector Machines (DSVM) [5] measure the similarity of each pair of classes using the Euclidean distance between their centroids. A top-down recursive method, called Divide-by-2 (DB2), is presented in [32] that employs a k-means clustering of the centroids of classes to iteratively divide the set of classes into two groups.…”