Structural Deep Clustering Network

Abstract

Clustering is a fundamental task in data analysis. Recently, deep clustering, which derives inspiration primarily from deep learning approaches, achieves state-of-the-art performance and has attracted considerable attention. Current deep clustering methods usually boost the clustering results by means of the powerful representation ability of deep learning, e.g., autoencoder, suggesting that learning an effective representation for clustering is a crucial requirement. The strength of deep clustering methods is to extract the useful representations from the data itself, rather than the structure of data, which receives scarce attention in representation learning. Motivated by the great success of Graph Convolutional Network (GCN) in encoding the graph structure, we propose a Structural Deep Clustering Network (SDCN) to integrate the structural information into deep clustering. Specifically, we design a delivery operator to transfer the representations learned by autoencoder to the corresponding GCN layer, and a dual self-supervised mechanism to unify these two different deep neural architectures and guide the update of the whole model. In this way, the multiple structures of data, from low-order to high-order, are naturally combined with the multiple representations learned by autoencoder. Furthermore, we theoretically analyze the delivery operator, i.e., with the delivery operator, GCN improves the autoencoder-specific representation as a high-order graph regularization constraint and autoencoder helps alleviate the over-smoothing problem in GCN. Through comprehensive experiments, we demonstrate that our propose model can consistently perform better over the state-of-the-art techniques.

A Deep Graph Structured Clustering Network

“…(1) Since we use deep GCN layers strategy, our experimental results are superior to SDCN [25] when γ = 0. Results indicate that when the value of γ is small, the graph reconstruction feature is insufficient.…”

“…However, it cannot be generalized. Deyu Bo et al [25] considers information about the features of the data itself and adds potential representations of its own features to the GCN layer for integrated learning, but ignoring the spatial topology information of the graph structure itself. Furthermore, GCN layer of all the above clustering methods is only a simple stack, and it leads to insufficient learning of complex structured information.…”

“…(8) SDCN [25]: It uses GCN and DNN auto-encoders to jointly construct a clustering network. And the selfsupervised module is proposed to update the parameters of graph convolution module and DNN auto-encoder module.…”

“…Therefore, the deep GCN layers in our proposed model not only perform the clustering task efficiently, but also consider whether oversmoothing can be alleviated. Here we compare our proposed method DGSCN with SDCN [25] and GAE [3] in Citeseer and Wiki dataset. The results are as follows in Fig.…”

CommDGI

“…GCN is used to encode node information, and a differentiable version of k-means clustering is used to learn the cluster partition. Structural Deep Clustering Network(SDCN) [5] applies a GCN module, consisting of multiple graph convolutional layers, to learn the GCN-specific representation. SDCN also applies an autoencoder module to learn the autoencoder-specific representation from the raw data and uses a dual self-supervised module to uniformly guide these two modules.…”