EIGAT: Incorporating global information in local attention for knowledge representation learning

Yu, Zhiwu; Feng, Huali; Zhou, Han; Yang, Yanruo; Chen, Xingyan; Xie, Rong; Zhuang, Fuzhen; Li, Qing

doi:10.1016/j.knosys.2021.107909

Cited by 13 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HRAN (Li, Liu, et al, 2021) takes full advantage of the heterogeneity of KG when aggregating neighbour information, but they ignore the overall structure. Unlike many models that focus only on information about 1‐hop neighbourhoods, EIGAT (Zhao et al, 2022) and GFA‐NN (Sadeghi et al, 2021) take into account the global characteristics of the entities, but still unable to perform cross‐relational information transfer. Within the recent years, most of the emerging KGE models utilize complex neural structures, such as tensor networks, graph convolutional networks, and transformers, to learn richer representations.…”

Section: Related Workmentioning

confidence: 99%

An attention‐based representation learning model for multiple relational knowledge graph

et al. 2023

View full text Add to dashboard Cite

Knowledge graph embedding models are used to learn low‐dimensional representations of entities and relations in knowledge graphs. In this paper, we propose Multi‐RAttE, an attention‐based learning method for multiple relational knowledge graph embedding representation, which divides the information transfer in the knowledge graph into cross‐relational information transfer and relation‐specific information transfer, and divides the embedding of knowledge graph entities into structural embedding and multi‐relational embedding for joint learning. To objectively analyse the performance of the Multi‐RAttE model, we select two typical datasets and different representative baseline models for experimental evaluation on several tasks such as link prediction, multi‐relation prediction and node classification. The experimental results show that the Multi‐RAttE model improves 8% over the state‐of‐the‐art model Composition‐based Multi‐Relational Graph Convolutional Networks (CompGCN) in terms of Hits@1 metric on the link prediction task on FB15k‐237 dataset; on the multi‐relation prediction task, the accuracy improves by 1.8% and 3.7% in the auc metric and F1 metric, respectively. The experimental results have proved that the Multi‐RAttE model can effectively perform the representation of multiple relations.

show abstract

Section: Related Workmentioning

confidence: 99%

An attention‐based representation learning model for multiple relational knowledge graph

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Dong et al [24] propose a graph node representation learning model by maximizing the mutual information between the current node and neighbor nodes. The EIGAT [25] algorithm proposes a method of incorporating global features into local attention for knowledge representation learning, which combines global features appropriately into the GAT model family by using scaled entity importance, which is computed by an attention-based global random walk strategy. The MEGNN [26] algorithm proposes a meta path extraction GNNs for heterogeneous graphs, which combine different bipartite graphs related to edge types into a new trainable graph structure.…”

Section: Related Workmentioning

confidence: 99%

A Network Representation Learning Model Based on Multiple Remodeling of Node Attributes

Zhang,

Cui,

et al. 2023

Mathematics

View full text Add to dashboard Cite

Current network representation learning models mainly use matrix factorization-based and neural network-based approaches, and most models still focus only on local neighbor features of nodes. Knowledge representation learning aims to learn low-dimensional dense representations of entities and relations from structured knowledge graphs, and most models use the triplets to capture semantic, logical, and topological features between entities and relations. In order to extend the generalization capability of the network representation learning models, this paper proposes a network representation learning algorithm based on multiple remodeling of node attributes named MRNR. The model constructs the knowledge triplets through the textual association relationships between nodes. Meanwhile, a novel co-occurrence word training method has been proposed. Multiple remodeling of node attributes can significantly improve the effectiveness of network representation learning. At the same time, MRNR introduces the attention mechanism to achieve the weight information for key co-occurrence words and triplets, which further models the semantic and topological features between entities and relations, and it makes the network embedding more accurate and has better generalization ability.

show abstract

“…These approaches can be divided into four categories: (i) translation-based models, which consider the translation operation between entity and relation embedding, such as TransE [10] and TransH [10]; (ii) factorization-bsaed models, which assume KG as a third-order tensor matrix, and the triple score can be carried out through matrix decomposition. Such as RESCAL [11], HOLE [12]; (iii) CNN-based models, which employ convolutional neural networks to determine the scores of triples, such as ConvE [13] and ConvKB [14]; and (iv) Graph neural network-based models, which extend convolution operations onto non-Euclidean graph structures, such as RGCN [15], KBGAT [16], EIGAT [17] and CompGCN [18].…”

Section: B Related Workmentioning

confidence: 99%

Embedding Knowledge Graphs with Semantic-Guided Walk

Tang¹,

Liu²,

Xu³

et al. 2022

Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Knowledge graph completion can complete knowledge by predicting missing facts, which is a increasingly hot research topic in knowledge graph construction. Prevalent approaches propose to embed knowledge graphs in a lowdimensional vector space and use these embedding to predict, but they neglect either semantic information or graph structures. We propose a new approach to knowledge graph completion named as ATTWALK, which learns embedding by exploiting both structural and semantic features of a knowledge graph. This is achieved by leveraging a key insight that an entities' embedding is influenced by its multi-hop neighbors', which can be further distinguished by their semantic importance to the entity. ATTWALK orchestrates a two-step workflow by first evaluating neighbors' semantic weights using graph attention networks for each entity, then exploring the entities' local structural features by performing a semantic weight guided walk. We evaluate ATTWALK by conducting extensive experiments, which show that ATTWALK outperforms 12 representative approaches on average across 3 publicly available datasets.

show abstract

EIGAT: Incorporating global information in local attention for knowledge representation learning

Cited by 13 publications

References 13 publications

An attention‐based representation learning model for multiple relational knowledge graph

An attention‐based representation learning model for multiple relational knowledge graph

A Network Representation Learning Model Based on Multiple Remodeling of Node Attributes

Embedding Knowledge Graphs with Semantic-Guided Walk

Contact Info

Product

Resources

About