Incorporating Global Information in Local Attention for Knowledge Representation Learning

Yu, Zhiwu; Zhou, Han; Xie, Rong; Zhuang, Fuzhen; Li, Qing; Liu, Ji

doi:10.18653/v1/2021.findings-acl.115

Cited by 11 publications

(6 citation statements)

References 59 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we will try to use better methods to utilize the knowledge graph structure, such as other knowledge graph embedding methods. As introduced in Section 2.3, some recent knowledge graph embedding methods such as HAKE [44], PairRE [45], DualE [46], and EIGAT [47] can better encode entities and relations in knowledge graphs, and theoretically they should further improve the performance of entity linking.…”

Section: Discussionmentioning

confidence: 99%

“…DualE universally models relations as the combination of a series of translation and rotation operations. EIGAT [47] allows correct incorporation of global information into the graph attention network (GAT) family of models by using scaled entity importance, which is computed by an attention-based global random walk algorithm. In order to focus on the importance of the knowledge graph structure for the entity linking task, the knowledge graph embedding method used in this article is the most basic TransE model.…”

Section: Knowledge Graph Embeddingmentioning

confidence: 99%

See 1 more Smart Citation

Improving Entity Linking by Introducing Knowledge Graph Structure Information

Feng

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Entity linking involves mapping ambiguous mentions in documents to the correct entities in a given knowledge base. Most of the current methods are a combination of local and global models. The local model uses the local context information around the entity mention to independently resolve the ambiguity of each entity mention. The global model encourages thematic consistency across the target entities of all mentions in the document. However, the known global models calculate the correlation between entities from a semantic perspective, ignoring the correlation information between entities in nature. In this paper, we introduce knowledge graphs to enrich the correlation information between entities and propose an entity linking model that introduces the structural information of the knowledge graph (KGEL). The model can fully consider the relations between entities. To prove the importance of the knowledge graph structure, extensive experiments are conducted on multiple public datasets. Results illustrate that our model outperforms the baseline and achieves superior performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Knowledge Graph Embeddingmentioning

confidence: 99%

Improving Entity Linking by Introducing Knowledge Graph Structure Information

Feng

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Zhang, Zhuang et al [4] transforms it into relation-level attention and entity-level attention. Zhao, Zhou et al [17] adds additional global attention mechanism. Fang, Wang et al [18] consider the impact of neighborhoods on representation learning of relations.…”

Section: Related Work a Link Predictionmentioning

confidence: 99%

“…Due to the graph structure of KG, it is natural to introduce graph convolutional networks (GCN) into link prediction. In addition, the emergence of graph attention networks (GAT) [3]- [5], which incorporate attention mechanisms into GCNs, achieve great improvement.…”

Section: Introductionmentioning

confidence: 99%

Few-Shot Link Prediction using Variational Heterogeneous Attention Networks

Tao,

Wang,

Luo

et al. 2023

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Few-shot link prediction is an important recent research direction in the field of knowledge graph. To learn better entity and relation representations, previous works utilize the neighborhood information of entities but they ignore the associated relations. More than this, the link invovles few-shot entities cannot be well linked by these methods. Therefore, this paper proposes a novel method that dynamically aggregates local neighbourhood information from entities and relations and introduces the information bottleneck principle to filter irrelevant features to better capture the feature of few-shot entities. Experimental results on three benchmark datasets demonstrate that our model outperforms other models and that our approach can effectively improve the model performance on those entities in the tail of the long-tailed distribution.

show abstract

“…Previous work generally uses one-hot encoding to represent entity knowledge, but using graph neural networks to encode structural knowledge and attribute entity representation for fine-grained modeling of entities, vocabulary, etc., allows the introduction of multi-angle feature knowledge [19]. Knowledge representation based on graph networks can flexibly use local, spatial, and other structural information, and can effectively address the polysemy of words [20]. Therefore, using graph networks to implement knowledge representation has become a hot research topic in recent years such as Ebisu, Balazevic Zhang, and other related scholars used graphs to accomplish different tasks respectively [21] [22] [23].…”

Section: Related Work a Graph Neural Network And Knowledge Representa...mentioning

confidence: 99%

MetaGNN-Based Medical Records Unstructured Specialized Vocabulary Few-Shot Representation Learning

Ling

Luo

Yu³

2022

IEEE Access

View full text Add to dashboard Cite

With the continuous breakthroughs in artificial intelligence technology, it has become easier to extract general-purpose knowledge using machine learning, but it is a challenging task to extract and learn small samples of knowledge in medical expertise. On the one hand, it is difficult to represent medical expertise entities, and on the other hand, the training samples of such expertise are small, and deep learning methods often require a large number of samples to complete the learning task. To this end, we proposes a graph network learning method for specialized vocabulary representation. Specifically, a contextual knowledge representation model based on graph meta-learning is proposed, which combines text, phrase, vocabulary, and other information to solve the problem of sparse data of medical electronic medical record entities that cannot be extracted and learned. In this method, a text-independent lexical representation learning method, a context-aware graph neural network, and a combined LSTM language model are used to model information from different perspectives as a way to learn semantic representations of professional discourse entities. The experimental results show that the accuracy of the method outperforms other similar methods and proves its effectiveness.

show abstract

Incorporating Global Information in Local Attention for Knowledge Representation Learning

Cited by 11 publications

References 59 publications

Improving Entity Linking by Introducing Knowledge Graph Structure Information

Improving Entity Linking by Introducing Knowledge Graph Structure Information

Few-Shot Link Prediction using Variational Heterogeneous Attention Networks

MetaGNN-Based Medical Records Unstructured Specialized Vocabulary Few-Shot Representation Learning

Contact Info

Product

Resources

About