Soft Marginal TransE for Scholarly Knowledge Graph Completion

Nayyeri, Mojtaba; Vahdati, Sahar; Lehmann, Jens; Yazdi, Hamed Shariat

doi:10.48550/arxiv.1904.12211

Cited by 4 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We generate 100 minibatches in each iteration. The hyperparameter corresponding to the score function is embedding dimension d. We add slack variables to the losses 4 and 6 to have soft margin as in (Nayyeri et al, 2019). The loss 6 is rewritten as follows (Nayyeri et al, 2019):…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding

Nayyeri,

Xu,

Yaghoobzadeh

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Knowledge graphs (KGs) represent world's facts in structured forms. KG completion exploits the existing facts in a KG to discover new ones. Translation-based embedding model (TransE) is a prominent formulation to do KG completion. Despite the efficiency of TransE in memory and time, it suffers from several limitations in encoding relation patterns such as symmetric, reflexive etc. To resolve this problem, most of the attempts have circled around the revision of the score function of TransE i.e., proposing a more complicated score function such as Trans(A, D, G, H, R, etc) to mitigate the limitations. In this paper, we tackle this problem from a different perspective. We show that existing theories corresponding to the limitations of TransE are inaccurate because they ignore the effect of loss function. Accordingly, we pose theoretical investigations of the main limitations of TransE in the light of loss function. To the best of our knowledge, this has not been investigated so far comprehensively. We show that by a proper selection of the loss function for training the TransE model, the main limitations of the model are mitigated. This is explained by setting upper-bound for the scores of positive samples, showing the region of truth (i.e., the region that a triple is considered positive by the model). Our theoretical proofs with experimental results fill the gap between the capability of translation-based class of embedding models and the loss function. The theories em-phasise the importance of the selection of the loss functions for training the models. Our experimental evaluations on different loss functions used for training the models justify our theoretical proofs and confirm the importance of the loss functions on the performance.

show abstract

Section: Methodsmentioning

confidence: 99%

“…(4) Condition (c) considers a triple to be positive if its residual vector lies inside a hyper-sphere with radius γ 1 . The optimization problem that satisfies the condition (c) is as follows (Nayyeri et al, 2019):…”

Section: Reinvestigation Of the Limitations Of Translation-based Embe...mentioning

confidence: 99%

Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding

Nayyeri,

Xu,

Yaghoobzadeh

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is represented as limited-based scoring loss illustrated in Figure 1. In this way, the scores of positive samples are forced to stay before the upper bound which signi cantly improves the performance of translation-based KGE models [2,21,29]. Zhou et al [29] revises the MRL by adding a term ([f r (h, t) − γ 1 ] + ) to limit maximum value of positive score:…”

Section: Limited-based Scoring Lossmentioning

confidence: 99%

“…A modi ed version of the two previous loss functions is introduced in our previous work [21]. is approach xes the upper-bound of positive samples (γ 1 ) and uses a sliding mechanism to move false negative samples towards positive samples, shown as So Margin in Figure 1.…”

Section: So Marginmentioning

confidence: 99%

Adaptive Margin Ranking Loss for Knowledge Graph Embeddings via a Correntropy Objective Function

Nayyeri,

Zhou,

Vahdati

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Translation-based embedding models have gained signi cant a ention in link prediction tasks for knowledge graphs. TransE is the primary model among translation-based embeddings and is wellknown for its low complexity and high e ciency. erefore, most of the earlier works have modi ed the score function of the TransE approach in order to improve the performance of link prediction tasks. Nevertheless, proven theoretically and experimentally, the performance of TransE strongly depends on the loss function. Margin Ranking Loss (MRL) has been one of the earlier loss functions which is widely used for training TransE. However, the scores of positive triples are not necessarily enforced to be su ciently small to ful ll the translation from head to tail by using relation vector (original assumption of TransE). To tackle this problem, several loss functions have been proposed recently by adding upper bounds and lower bounds to the scores of positive and negative samples.Although highly e ective, previously developed models su er from an expansion in search space for selection of the hyperparameters (in particular the upper and lower bounds of scores) on which the performance of the translation-based models is highly dependent. In this paper, we propose a new loss function dubbed Adaptive Margin Loss (AML) for training translation-based embedding models.e formulation of the proposed loss function enables an adaptive and automated adjustment of the margin during the learning process. erefore, instead of obtaining two values (upper bound and lower bound), only the center of a margin needs to be determined. During learning, the margin is expanded automatically until it converges. In our experiments on a set of standard benchmark datasets including Freebase and WordNet, the e ectiveness of AML is con rmed for training TransE on link prediction tasks.

show abstract

“…Because the network security knowledge graph is sparse, representation learning in the network security field also faces insufficient structural information [15,16]. It has been shown that there are three main issues in current representation learning.…”

Section: Introductionmentioning

confidence: 99%

A Knowledge Graph Completion Algorithm Based on the Fusion of Neighborhood Features and vBiLSTM Encoding for Network Security

Zhang,

Wang,

Han

et al. 2024

Electronics

View full text Add to dashboard Cite

Knowledge graphs in the field of network security can integrate diverse, heterogeneous, and fragmented network security data, further explore the relationships between data, and provide support for deep analysis. Currently, there is sparse security information in the field of network security knowledge graphs. The limited information provided by traditional text encoding models leads to insufficient reasoning ability, greatly restricting the development of this field. Starting from text encoding, this paper first addresses the issue of the inadequate capabilities of traditional models using a deep learning model for assistance. It designs a vBiLSTM model based on a word2vec and BiLSTM combination to process network security texts. By utilizing word vector models to retain semantic information in entities and extract key features to input processed data into BiLSTM networks for extracting higher-level features that better capture and express their deeper meanings, this design significantly enhances understanding and expression capabilities toward complex semantics in long sentences before inputting final feature vectors into the KGC-N model. The KGC-N model uses feature vectors combined with graph structure information to fuse forward and reverse domain features and then utilizes a Transformer decoder to decode predictions and complete missing information within the network security knowledge map. Compared with other models using evaluation metrics such as MR, MRR demonstrates that employing our proposed method effectively improves performance on completion tasks and increases comprehension abilities toward complex relations, thereby enhancing accuracy and efficiency when completing knowledge graphs.

show abstract

Soft Marginal TransE for Scholarly Knowledge Graph Completion

Cited by 4 publications

References 12 publications

Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding

Toward Understanding The Effect Of Loss function On Then Performance Of Knowledge Graph Embedding

Adaptive Margin Ranking Loss for Knowledge Graph Embeddings via a Correntropy Objective Function

A Knowledge Graph Completion Algorithm Based on the Fusion of Neighborhood Features and vBiLSTM Encoding for Network Security

Contact Info

Product

Resources

About