Regularized Discriminant in the Setting of Fuzzy c-Means Classifier

Abstract

A fuzzy c-means classifier derived from a viewpoint of iteratively reweighted least square techniques (IRLS-FCM) has been proposed, in which membership functions are variously chosen and parameterized. This paper focuses on the postsupervised classifier design and three kinds of regularization methods for classification are addressed: 1) the exponent of membership function or weights in entropy term in the FCM clustering, 2) the modification of covariance matrices in defining Mahalanobis distances and 3) the designing of intermediate classification rules between linear and quadratic in the regularized discriminant analysis (RDA).We test the efficiency of these regularization methods in the IRLS-FCM classifier with regard to its performance and data set compression ratio, and show the best parameter values. Among the three regularization approaches, the improvement in classification performances is achieved mostly by the methods 1) and 2). Experiments on several well-known benchmark data sets, shows that the FCM classifier using a newly defined membership function outperforms well-established prototypebased methods, i.e., k-nearest neighbor classier (k-NN) and learning vector quantization (LVQ). Also concerning storage requirements and classifcation speed, the IRLS-FCM classifier gives a good performance and efficiency.

Fuzzy c-Means Classifier for Relational Data

“…For Breast cancer data, Euclidean distance, (i.e., p=0, c=3), was used and the classification decision was made by the maximum of cluster membership values in [8]. The result for this case by RFCMC was 2.79 ± 0.07, which is slightly better than the result in Table II in which Mahalanobis distances are used (p = 1) and the decision was made by the sum of memberships.…”

“…The parameters of RFCMC are optimized by trial and error using 10-CV with a default partition and evaluated by 10 separate runs of 10-CV with random partitions. The results of FCMC on object data and optimized by trial and error are reported in [8].…”

“…v i is a cluster centroid. The dimension of x and v is m. v i is a centroid for the ith cluster given by (8).…”

“…From the above consideration, a generalized FCM clustering is proposed and applied to a post-supervised classifier design and is called IRLS-FCM classifier (FCMC) [7], [8], [9]. The classifier with deterministic initialization clearly outperforms many well established classifiers [10].…”

FCM classifier for high-dimensional data

“…The classifier based on the GMM clustering is called a neural network [5] or the Gaussian mixture classifier (GMC). The classifier based on the FCM clustering is called the fuzzy c-means classifier (FCMC) [6], [7], [8]. For the detail derivation of the algorithm and the classification performance, see [9](WCCI'08).…”