A Multiobjective Simultaneous Learning Framework for Clustering and Classification

Abstract

Traditional pattern recognition involves two tasks: clustering learning and classification learning.Clustering result can enhance the generalization ability of classification learning, while the class information can improve the accuracy of clustering learning. Hence, both learning methods can complement each other. To fuse the advantages of both learning methods together, many existing algorithms have been developed in a sequential fusing way by first optimizing the clustering criterion and then the classification criterion associated with the obtained clustering results. However, such kind of algorithms naturally fails to achieve the simultaneous optimality for two criteria, and thus have to sacrifice either the clustering performance or the classification performance. To overcome that problem, in this paper, we present a multi-objective simultaneous learning framework (named MSCC) for both clustering and classification learning. MSCC utilizes multiple objective functions to formulate the clustering and classification problems, respectively, and more importantly it employs the Bayesian theory to make these functions all only dependent on a set of the same parameters, i.e., clustering centers which play a role of the bridge connecting the clustering and classification learning. By simultaneously optimizing the clustering centers embedded in these functions, not only the effective clustering performance but also the promising classification performance can be simultaneously attained. Furthermore, from the multiple Pareto-optimality solutions obtained in MSCC, we can get an interesting observation that there is complementarity to great extent between clustering and classification learning processes. Empirical results on both synthetic and real data sets demonstrate the effectiveness and potential of MSCC.

Improving Classification Performance of Fully Connected Layers by Fuzzy Clustering in Transformed Feature Space

“…For the reason of preferring a cross entropy loss rather than directly using the MSE loss as the clustering cost, please refer to "Algorithmic Details of the Proposed Method" part of this study. Cross entropy calculation is given in Equation (7). Since all elements of y are zero, Equation ( 7) simplifies to Equation ( 8).…”

“…The result of the experiments, a robust framework which is capable of clustering and classifying a dataset simultaneously has been asserted as the output of their study. Additionally, in Cai et al [7], the authors advanced their framework to employ multiple objective functions for clustering and classification rather than a single one. Qian et al [8] addressed the problem of high complexity in the objective function that is suggested by [6,7] and proposed a new framework that exploits cluster structure representations instead of cluster posterior probabilities of classes to relate clusters and classes.…”

“…Additionally, in Cai et al [7], the authors advanced their framework to employ multiple objective functions for clustering and classification rather than a single one. Qian et al [8] addressed the problem of high complexity in the objective function that is suggested by [6,7] and proposed a new framework that exploits cluster structure representations instead of cluster posterior probabilities of classes to relate clusters and classes. Thanks to decreased complexity and continuously differentiable forms of the objective function, a block coordinate descent algorithm was employed as the optimizer in their method, resulting in a more efficient framework.…”