Fast Hierarchy Preserving Graph Embedding via Subspace Constraints

Chen, Xu; Du, Lun; Wang, Yun; Long, Qingqing; Xie, Kun

doi:10.1109/icassp39728.2021.9414919

Cited by 5 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Graph Neural Networks (GNNs) have been popular for modeling graph data [3,5,9,35,38]. GCN [19] proposed to use graph convolution based on neighborhood aggregation.…”

Section: Related Work 61 Graph Representation Learningmentioning

confidence: 99%

Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach

Bi¹,

Du²,

Fu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Graph Neural Networks (GNNs) are popular machine learning methods for modeling graph data. A lot of GNNs perform well on homophily graphs while having unsatisfied performance on heterophily graphs. Recently, some researchers turn their attentions to designing GNNs for heterophily graphs by adjusting message passing mechanism or enlarging the receptive field of the message passing. Different from existing works that mitigate the issues of heterophily from model design perspective, we propose to study heterophily graphs from an orthogonal perspective by rewiring the graph structure to reduce heterophily and making the traditional GNNs perform better. Through comprehensive empirical studies and analysis, we verify the potential of the rewiring methods. To fully exploit its potential, we propose a method named Deep Heterophily Graph Rewiring (DHGR) to rewire graphs by adding homophilic edges and pruning heterophilic edges. The detailed way of rewiring is determined by comparing the similarity of label/feature-distribution of node neighbors. Besides, we design a scalable implementation for DHGR to guarantee a high efficiency. DHRG can be easily used as a plug-in module, i.e., a graph pre-processing step, for any GNNs, including both GNN for homophily and heterophily, to boost their performance on the node classification task. To the best of our knowledge, it is the first work studying graph rewiring for heterophily graphs. Extensive experiments on 11 public graph datasets demonstrate the superiority of our proposed methods.

show abstract

“…Graph Neural Networks (GNNs) have been popular for modeling graph data [3,5,9,35,38]. GCN [19] proposed to use graph convolution based on neighborhood aggregation.…”

Section: Related Work 61 Graph Representation Learningmentioning

confidence: 99%

Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach

Bi¹,

Du²,

Fu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The preliminary version of this work was presented at the conference ICASSP 2021 [9]. This paper makes the following additional contributions:…”

Section: Introductionmentioning

confidence: 99%

A subspace constraint based approach for fast hierarchical graph embedding

Chen

et al. 2023

World Wide Web

Self Cite

View full text Add to dashboard Cite

Hierarchy network, as a type of complex graphs, is widely used in many application scenarios such as social network analysis in web, human resource analysis in e-government, and product recommendation in ecommerce. Hierarchy preserving network embedding is a representation learning method that project nodes into feature space by preserving the hierarchy property of networks. Recently, researches on network embedding are devoted to mining hierarchical structures and profit a lot form it. Among these works, SpaceNE stands out of preserving hierarchy with the help of subspace constraint on the hierarchical subspace system. However, like all other existing works, SpaceNE is based on transductive learning method and is hard to generalize to new nodes. Besides, they have high time complexity and hard to be scalable to large-scale networks. This paper proposes an inductive method, FastHGE to learn node representation more efficiently and generalize to new nodes more easily. As SpaceNE, a hierarchy network is embedded into a hierarchical subspace tree. For upper communities, we exploit transductive learning by preserving inner-subspace proximity of subspace from the same ancestor. For extending to new nodes, we adopt inductive learning to learn representations of leaf nodes. The overall representation of a node is retrieved A Subspace Constraint based Approach for Fast Hierarchical Graph Embedding by concatenating the embedding vectors of all its ancestor communities and the leaf node. By learning the basis vectors of subspace, the computing cost is alleviated from updating many parameters of projection matrices as in SpaceNE. The performance evaluation experiments show that FastHGE outperforms with much fast speed and the same accuracy. For example, in the node classification, FastHGE is nearly 30 times faster than SpaceNE. The source code of FastHGE is available online.

show abstract

Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective

Chen

Gao

et al. 2022

Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Self Cite

View full text Add to dashboard Cite

Network Embedding aims to learn a function mapping the nodes to Euclidean space contribute to multiple learning analysis tasks on networks. However, both the noisy information behind the realworld networks and the overfitting problem negatively impact the quality of embedding vectors. To tackle these problems, researchers utilize Adversarial Perturbations on Parameters (APP) and achieve state-of-the-art performance. Unlike the mainstream methods introducing perturbations on the network structure or the data feature, Adversarial Training for Network Embedding (AdvTNE) adopts APP to directly perturb the model parameters, thus providing a new chance to understand the mechanism behind it. In this paper, we explain APP theoretically from an optimization perspective. Considering the Power-law property of networks and the optimization objective, we analyze the reason for its remarkable results on network embedding. Based on the above analysis and the Sigmoid saturation region problem, we propose a new Sine-base activation to enhance the performance of AdvTNE. We conduct extensive experiments on four real networks to validate the effectiveness of our method in node classification and link prediction. The results demonstrate that our method is competitive with state-of-the-art methods. CCS CONCEPTS• Networks → Topology analysis and generation; Physical topologies; • Information systems → Network data models; • Mathematics of computing → Mathematical optimization.

show abstract

Fast Hierarchy Preserving Graph Embedding via Subspace Constraints

Cited by 5 publications

References 16 publications

Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach

Make Heterophily Graphs Better Fit GNN: A Graph Rewiring Approach

A subspace constraint based approach for fast hierarchical graph embedding

Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective

Contact Info

Product

Resources

About