Learning hyperbolic attention-based embeddings for link prediction in knowledge graphs

Zeb, Adnan; Haq, Anwar Ul; Chen, Junde; Lei, Zhenfeng; Zhang, Defu

doi:10.1016/j.knosys.2021.107369

Cited by 21 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, TransE encodes entities and relations to linear space and TransR [22] expands the single linear space to a set of relation-speciic linear spaces. he embedding space can also be complex space [39], multi-dimensional Gaussian distribution space [43], hyperbolic space [49] and manifold space [16]. RotatE [37] maps the representation vectors to complex space and utilizes rotation operation of complex space to describe relation information between entities.…”

Section: Knowledge Graph Embeddingmentioning

confidence: 99%

Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion

Liang

Zhu

Zhang

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Knowledge graphs often suffer from incompleteness, and knowledge graph completion (KGC) aims at inferring the missing triplets through knowledge graph embedding from known factual triplets. However, most existing knowledge graph embedding methods only use the relational information of knowledge graph and treat the entities and relations as IDs with simple embedding layer, ignoring the multi-modal information among triplets, such as text descriptions, images, etc. In this work, we propose a novel network to incorporate different modal information with graph structure information for more precise representation of multi-modal knowledge graph, termed as hyper-node relational graph attention (HRGAT) network. In HRGAT, we use low-rank multi-modal fusion to model the intra-modality and inter-modality dynamics, which transforms the original knowledge graph to a hyper-node graph. Then, relational graph attention (RGAT) network is used, which contains relation-specific attention and entity-relation fusion operation to capture the graph structure information. Finally, we aggregate the updated multi-modal information and graph structure information to generate the final embeddings of knowledge graph to achieve KGC. By exploring multi-modal information and graph structure information, HRGAT embraces faster convergence speed and achieves the state-of-the-art for KGC on the standard datasets. Implementation code is available at https://github.com/broliang/HRGAT.

show abstract

Section: Knowledge Graph Embeddingmentioning

confidence: 99%

Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion

Liang

Zhu

Zhang

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…HGCN [25] further combines hyperbolic representation learning with the message passing network, projecting the node representations from the hyperbolic space to the tangent space for message aggregation. Based on the success of the above studies, [26,27,28,29] find that the hyperbolic space can well fit the hierarchical and logical patterns in KGs, and thus achieve a series of competitive performances through the high-fidelity and concise representation learning. Similarly, user-item graphs also exhibit the characteristics of a hierarchical and power-law distribution, and [30,31] therefore demonstrate the significance of combining hyperbolic geometry with recommendation tasks.…”

Section: Knowledge-enhanced Collaborative Filteringmentioning

confidence: 99%

Lorentz Equivariant Model for Knowledge-Enhanced Hyperbolic Collaborative Filtering

Bosong¹,

Yu²,

Xie³

et al. 2023

Preprint

View full text Add to dashboard Cite

Introducing prior auxiliary information from the knowledge graph (KG) to assist the user-item graph can improve the comprehensive performance of the recommender system. Many recent studies show that the ensemble properties of hyperbolic spaces fit the scale-free and hierarchical characteristics exhibited in the above two types of graphs well. However, existing hyperbolic methods ignore the consideration of equivariance, thus they cannot generalize symmetric features under given transformations, which seriously limits the capability of the model. Moreover, they cannot balance preserving the heterogeneity and mining the high-order entity information to users across two graphs. To fill these gaps, we propose a rigorously Lorentz group equivariant knowledgeenhanced collaborative filtering model (LECF). Innovatively, we jointly update the attribute embeddings (containing the high-order entity signals from the KG) and hyperbolic embeddings (the distance between hyperbolic embeddings reveals the recommendation tendency) by the LECF layer with Lorentz Equivariant Transformation. Moreover, we propose Hyperbolic Sparse Attention Mechanism to sample the most informative neighbor nodes. Lorentz equivariance is strictly maintained throughout the entire model, and enforcing equivariance is proven necessary experimentally. Extensive experiments on three real-world benchmarks demonstrate that LECF remarkably outperforms state-of-the-art methods.

show abstract

“…COMPGCN (Vashishth et al, 2019) introduces the entity-relation composition to leverage the combination of relations and nodes during the message transmission. Zeb et al (2021) propose an encoder-decoder framework based on the relation hyperbolic graph neural network (HyperGEL), which proved effective. Similarly, Zhang et al (2022) utilize the graph attention network enhanced with association rules to improve the performance of knowledge inference.…”

Section: Related Work 21 Structure-based Knowledge Graph Completionmentioning

confidence: 99%

“…Compared with other related studies of knowledge graph completion, the proposed model has the following advantages. First, compared with methods based on network structure such as PairRE (Chao et al, 2021), DMACM (Huang et al, 2021), RNNLogic (Qu et al, 2021) and HyperGEL (Zeb et al, 2021), the EDA-KGC model has significantly better performance due to the external information. Second, compared with methods that leverage entity description, such as DKRL (Xie et al, 2016b) and EDGE (Zhou et al, 2019), the proposed model uses the comprehensive integration of entity description and network structure to capture the potential semantic information.…”

Section: Fb15k-237 Mrrmentioning

confidence: 99%

A knowledge graph completion model integrating entity description and network structure

Zhang

et al. 2022

AJIM

View full text Add to dashboard Cite

PurposeIn recent years, knowledge graph completion has gained increasing research focus and shown significant improvements. However, most existing models only use the structures of knowledge graph triples when obtaining the entity and relationship representations. In contrast, the integration of the entity description and the knowledge graph network structure has been ignored. This paper aims to investigate how to leverage both the entity description and the network structure to enhance the knowledge graph completion with a high generalization ability among different datasets.Design/methodology/approachThe authors propose an entity-description augmented knowledge graph completion model (EDA-KGC), which incorporates the entity description and network structure. It consists of three modules, i.e. representation initialization, deep interaction and reasoning. The representation initialization module utilizes entity descriptions to obtain the pre-trained representation of entities. The deep interaction module acquires the features of the deep interaction between entities and relationships. The reasoning component performs matrix manipulations with the deep interaction feature vector and entity representation matrix, thus obtaining the probability distribution of target entities. The authors conduct intensive experiments on the FB15K, WN18, FB15K-237 and WN18RR data sets to validate the effect of the proposed model.FindingsThe experiments demonstrate that the proposed model outperforms the traditional structure-based knowledge graph completion model and the entity-description-enhanced knowledge graph completion model. The experiments also suggest that the model has greater feasibility in different scenarios such as sparse data, dynamic entities and limited training epochs. The study shows that the integration of entity description and network structure can significantly increase the effect of the knowledge graph completion task.Originality/valueThe research has a significant reference for completing the missing information in the knowledge graph and improving the application effect of the knowledge graph in information retrieval, question answering and other fields.

show abstract

Learning hyperbolic attention-based embeddings for link prediction in knowledge graphs

Cited by 21 publications

References 23 publications

Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion

Hyper-node Relational Graph Attention Network for Multi-modal Knowledge Graph Completion

Lorentz Equivariant Model for Knowledge-Enhanced Hyperbolic Collaborative Filtering

A knowledge graph completion model integrating entity description and network structure

Contact Info

Product

Resources

About