Analyzing Heterogeneous Networks With Missing Attributes by Unsupervised Contrastive Learning

He, Dongxiao; Liang, Chundong; Huo, Cuiying; Feng, Zhiyong; Jin, Dayong; Yang, Lijian; Zhang, Weixiong

doi:10.1109/tnnls.2022.3149997

Cited by 15 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(1) Focusing on a network, the link prediction accuracy rises and then falls as the L value increases. For example, for the DBLP dataset, the ACC value increases rapidly in the range [1,2]. Then, in the range [2,4], the ACC values stabilize.…”

Section: Parameter Sensitivitymentioning

confidence: 99%

“…This indicates that too small L values cannot fully capture the intra-type features of nodes in the network, and too large L leads to capturing imprecise intra-type features. Specifically, the relative stability range of the parameter L is [1,3] on Amazon, [2,4] on DBLP and Yelp. (2) Comparing different datasets, the window length of the Yelp and DBLP datasets should be larger than that of the Amazon dataset.…”

Section: Parameter Sensitivitymentioning

confidence: 99%

“…In the real world, many systems (such as traffic systems and social systems) can be abstracted into heterogeneous information networks (HINs) with different node types and link types [1]. However, as HINs grow in size, complex coupled network data cannot cope with the real-time demands of downstream network analysis tasks [2].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Robust and fast representation learning for heterogeneous information networks

Lei

Chen

et al. 2023

Front. Phys.

View full text Add to dashboard Cite

Network representation learning is an important tool that can be used to optimize the speed and performance of downstream analysis tasks by extracting latent features of heterogeneous networks. However, in the face of new challenges of increasing network size, diverse latent features, and unseen network noise, existing representation models need to be further optimized. In this paper, a robust and fast representation learning model is proposed for heterogeneous networks, called RFRL. First, the global features of a heterogeneous network are divided into multiple intra-type local features and inter-type local features, and a type-aware biased sampling is designed to generate training samples for each local feature. Second, a node-type-aware and a link-type-aware shallow representation strategy are used to learn intra-type features and inter-type features respectively. This enables the model to achieve good performance while having high speed through the divide-and-conquer learning process and shallow learning model, thus coping with increasing network size and latent feature diversity. Finally, adversarial learning is used to integrate the above two representation strategies to address unseen network noise and enhance the robustness of representation learning. Extensive experiments on three network analysis tasks and three public datasets demonstrate the good performance of our RFRL model.

show abstract

Section: Parameter Sensitivitymentioning

confidence: 99%

Section: Parameter Sensitivitymentioning

confidence: 99%

See 1 more Smart Citation

Robust and fast representation learning for heterogeneous information networks

Lei

Chen

et al. 2023

Front. Phys.

View full text Add to dashboard Cite

show abstract

“…Recently, clustering with attribute-missing data has attracted significant attention from researchers (Liu 2021;Jin et al 2021;He et al 2022;Cui et al 2022;Rossi et al 2022;Tu et al 2022;Yoo et al 2022;Xu et al 2022;Jin et al 2023). For example, SAT (Chen et al 2022) and CSAT (Li et al 2022b) perform distribution matching between the attribute embedding and the structure embedding to estimate missing attributes for clustering.…”

Section: Clustering With Attribute-missing Datamentioning

confidence: 99%

Attribute-Missing Graph Clustering Network

Tu,

Guan,

Zhou

et al. 2024

AAAI

View full text Add to dashboard Cite

Deep clustering with attribute-missing graphs, where only a subset of nodes possesses complete attributes while those of others are missing, is an important yet challenging topic in various practical applications. It has become a prevalent learning paradigm in existing studies to perform data imputation first and subsequently conduct clustering using the imputed information. However, these ``two-stage" methods disconnect the clustering and imputation processes, preventing the model from effectively learning clustering-friendly graph embedding. Furthermore, they are not tailored for clustering tasks, leading to inferior clustering results. To solve these issues, we propose a novel Attribute-Missing Graph Clustering (AMGC) method to alternately promote clustering and imputation in a unified framework, where we iteratively produce the clustering-enhanced nearest neighbor information to conduct the data imputation process and utilize the imputed information to implicitly refine the clustering distribution through model optimization. Specifically, in the imputation step, we take the learned clustering information as imputation prompts to help each attribute-missing sample gather highly correlated features within its clusters for data completion, such that the intra-class compactness can be improved. Moreover, to support reliable clustering, we maximize inter-class separability by conducting cost-efficient dual non-contrastive learning over the imputed latent features, which in turn promotes greater graph encoding capability for clustering sub-network. Extensive experiments on five datasets have verified the superiority of AMGC against competitors.

show abstract

“…Zhu et al [22] addressed a contrastive representation method to enhance a reinforcement learning framework, which considered the correlation among consecutive inputs, and jointly trained the CNN encoder and Transformer through a contrastive learning process, in order to reconstruct features based on context frames. He et al [23] proposed a graph contrastive learning model, in which a contrastive learning scheme was developed to train the attribute completion and representation learning in an unsupervised heterogeneous framework, aiming to handle the missing attributes and jointly learn the embeddings of nodes and attributes.…”

Section: B Contrastive Learningmentioning

confidence: 99%

Personalized Federated Learning With Model-Contrastive Learning for Multi-Modal User Modeling in Human-Centric Metaverse

Zhou,

Yang,

Zheng

et al. 2024

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

With the flourish of digital technologies and rapid development of 5G and beyond networks, Metaverse has become an increasingly hotly discussed topic, which offers users with multiple roles for diversified experience interacting with virtual services. How to capture and model users' multi-platform or cross-space data/behaviors become essential to enrich people with more realistic and immersed experience in Metaverseenabled smart applications over 5G and beyond networks. In this study, we propose a Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) framework, which may efficiently enhance the communication and interaction in humancentric Metaverse environments by making use of the largescale, heterogeneous, and multi-modal Metaverse data. Differing from the conventional Federated Learning (FL) architecture, a multi-center aggregation structure to learn multiple global models based on the changes of dynamically updated local model weights, is developed in global, while a hierarchical neural network structure which includes a personalized module and a federated module to tackle both issues on data heterogeneity and model heterogeneity, is designed in local, so as to enhance the performance of PFL with unique characteristics of Metaverse data. In particular, a two-stage iterative clustering algorithm with a more precise initialization is developed to facilitate the personalized global aggregation with dynamically updated multiple aggregation centers. A personalized multi-modal fusion network is constructed to greatly reduce the computational cost and feature dimensions from the high-dimensional heterogeneous inputs for more efficient cross-modal fusion, based on a hierarchical shift-window attention mechanism and a newly designed bridge attention mechanism. A MCL scheme is then incorporated to speed up the model convergence with less communication

show abstract

Analyzing Heterogeneous Networks With Missing Attributes by Unsupervised Contrastive Learning

Cited by 15 publications

References 37 publications

Robust and fast representation learning for heterogeneous information networks

Robust and fast representation learning for heterogeneous information networks

Attribute-Missing Graph Clustering Network

Personalized Federated Learning With Model-Contrastive Learning for Multi-Modal User Modeling in Human-Centric Metaverse

Contact Info

Product

Resources

About