Explaining Link Prediction Systems based on Knowledge Graph Embeddings

Rossi, Andrea; Firmani, Donatella; Merialdo, Paolo; Teofili, Tommaso

doi:10.1145/3514221.3517887

Cited by 25 publications

(10 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other evaluation components may be added, such as the storage and computational requirements of KGEMs [ Wang et al , 2021c, Portisch et al , 2020 and the environmental impact of their training and testing procedure [ Peng et al , 2021 ]. Furthermore, the explainability of KGEMs is another dimension that deserves great attention [ Zhang et al , 2020b, Rossi et al , 2022.…”

Section: Discussionmentioning

confidence: 99%

Sem@$K$: Is my knowledge graph embedding model semantic-aware?

Hubert¹,

Monnin²,

Brun³

et al. 2023

Preprint

View full text Add to dashboard Cite

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constrains. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study of KGEM semantic awareness. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offer different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.

show abstract

Section: Discussionmentioning

confidence: 99%

Sem@$K$: Is my knowledge graph embedding model semantic-aware?

Hubert¹,

Monnin²,

Brun³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Other evaluation components may be added, such as the storage and computational requirements of KGEMs [51,69] and the environmental impact of their training and testing procedure [50]. Furthermore, the explainability of KGEMs is another dimension that deserves great attention [55,83].…”

Section: Rq3: Does the Evaluation Of Kgem Semantic Awareness Offers D...mentioning

confidence: 99%

Sem@K: Is my knowledge graph embedding model semantic-aware?

Hubert,

Monnin,

Brun

et al. 2023

View full text Add to dashboard Cite

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.

show abstract

“…Information can be obtained from a graph through deductive (e.g., logical rules) and inductive methods (e.g., as continuous graph embeddings) [37]. Both methods need to be transparent to the user [13,81] to be trustworthy.…”

Section: The Kg Lifecyclementioning

confidence: 99%

Towards Explainable Automatic Knowledge Graph Construction with Human-in-the-Loop

Zhang

Meroño‐Peñuela

Simperl

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Knowledge graphs are important in human-centered AI because of their ability to reduce the need for large labelled machine-learning datasets, facilitate transfer learning, and generate explanations. However, knowledge-graph construction has evolved into a complex, semi-automatic process that increasingly relies on opaque deep-learning models and vast collections of heterogeneous data sources to scale. The knowledge-graph lifecycle is not transparent, accountability is limited, and there are no accounts of, or indeed methods to determine, how fair a knowledge graph is in the downstream applications that use it. Knowledge graphs are thus at odds with AI regulation, for instance the EU’s upcoming AI Act, and with ongoing efforts elsewhere in AI to audit and debias data and algorithms. This paper reports on work in progress towards designing explainable (XAI) knowledge-graph construction pipelines with human-in-the-loop and discusses research topics in this space. These were grounded in a systematic literature review, in which we studied tasks in knowledge-graph construction that are often automated, as well as common methods to explain how they work and their outcomes. We identified three directions for future research: (i) tasks in knowledge-graph construction where manual input remains essential and where there may be opportunities for AI assistance; (ii) integrating XAI methods into established knowledge-engineering practices to improve stakeholder experience; as well as (iii) evaluating how effective explanations genuinely are in making knowledge-graph construction more trustworthy.

show abstract

Explaining Link Prediction Systems based on Knowledge Graph Embeddings

Cited by 25 publications

References 43 publications

Sem@$K$: Is my knowledge graph embedding model semantic-aware?

Sem@$K$: Is my knowledge graph embedding model semantic-aware?

Sem@K: Is my knowledge graph embedding model semantic-aware?

Towards Explainable Automatic Knowledge Graph Construction with Human-in-the-Loop

Contact Info

Product

Resources

About