An Image Clustering Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids and MMD Distance

Abstract

In this paper, we propose an end-to-end image clustering auto-encoder algorithm: ICAE. The algorithm uses PEDCC (Predefined Evenly-Distributed Class Centroids) as the clustering centers of the images, which ensures the inter-class distance of latent features is maximal, and adds data distribution constraint, data augmentation constraint, autoencoder reconstruction loss constraint and latent features plus noise constraint to improve clustering performance. Specifically, we perform one-to-one data augmentation such as rotation, shear, and shift before data is input to the encoder to learn the more effective features. The data and the enhanced data are simultaneously input into the auto-encoder to obtain latent features and augmented latent features whose similarity are constrained by an augmentation loss. Then, making use of the MMD distance, we combine the latent features and augmented latent features to make their distribution close to the PEDCC distribution (uniform distribution between classes, Dirac distribution within the class) to further learn the features used for clustering. At the same time, the MSE of the original input image and reconstructed image is used as reconstruction constraint, and the noise is added to the latent features to build generalization constraint to improve the generalization ability. Finally, extensive experiments on three common datasets MNIST, Fashion-MNIST, COIL20 are conducted. The experimental results show that the algorithm has achieved the best clustering results so far, and also has good generalization ability. In addition, we can use the pre-defined PEDCC class centers, and the decoding module of the auto-encoder to clearly generate the samples of each class. The code can be downloaded at xxx!

A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids

“…When the equilibrium state is reached, the n points on the hypersphere can be farthest apart. The detail of algorithm implementation is visible in [13].…”

“…In this paper, PEDCC proposed in CSAE (Zhu qiuyu et al, 2019) [13] is used to generate the class centroids of the evenly distributed normalized weight, which is called PEDCC weights. We replace the weight of the classification linear layer with PEDCC weights in CNNs, and the PEDCC weights are solidified during training to maximize the inter-class distance.…”

Semi-supervised learning method based on predefined evenly-distributed class centroids

“…However, if the inter-class distance is small, the accuracy of classification will be reduced. PEDCC is proposed based on the hypersphere charge model [20]. Due to mutual exclusion of charges, in equilibrium, n charges will be evenly distributed on the hypersphere, and the distance between points is the farthest.…”

Dual Supervised Autoencoder Based Trajectory Classification Using Enhanced Spatio-Temporal Information

“…As a result, we have a new data form that is presented as F Ss. In order to better extract the time information, we calculate RP for each F S. Inspired by the PEDCC [26] algorithm and Deep Embedded Clustering (DEC) [25] model, we introduce a CNN based supervised autoencoder that can learn the high-level features well and leverage them as much as possible. Besides, we also make the network accept both F Ss and RPs as inputs by extending the network to dual autoencoder, thus forming the Dual-CSA.…”

“…To effectively interpret the time information inside MFs and AFs, we develop a method to encode them as RPs. 2) We propose a cluster centroids aware supervised autoencoder that can make the high-level features (latent embedding) of the same class of samples as close as possible to PCC, which is mainly benefited from a loss function we improved from the one proposed in [25], and this improvement is based on the Predefined Evenly-Distributed Class Centroids (PEDCC) algorithm [26]. 3) A neural network consisting of two autoencoders is designed, one of which learns the high-level features of RPs and the other learns the high-level features of AFs and MFs.…”