Methods for Intrinsic Evaluation of Links in the Web of Data

Sarasua, Cristina; Staab, Steffen; Thimm, Matthias

doi:10.1007/978-3-319-58068-5_5

“…Measuring this information, and specifically how much information is gained by connecting two knowledge graphs together is a major aspect of this work. Here, we were strongly influenced by the work of Sarasua et al [19], who investigate the value of links between different data sets of a knowledge graph. To do so, statistics (based on [1]), such as the net amount of links, the average number of incoming and outgoing links per data set and number of entities linked by a source, are calculated.…”

Section: Related Workmentioning

confidence: 99%

“…In the second step, the intersection of the graphs is determined. If the parties still want to continue after this step, a set of metrics (based on [19]) is computed, while still maintaining the graphs secret. The outcome of the measures should give the Buyer an indication on whether the graph is interesting, but to determine the quality, actual data needs to be transferred (which could involve monetary exchange).…”

Section: Approachmentioning

confidence: 99%

Secure Evaluation of Knowledge Graph Merging Gain

Leandro¹,

Cochez²,

Heitmann³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

Finding out the differences and commonalities between the knowledge of two parties is an important task. Such a comparison becomes necessary, when one party wants to determine how much it is worth to acquire the knowledge of the second party, or similarly when two parties try to determine, whether a collaboration could be beneficial. When these two parties cannot trust each * Most of the work on this paper has been done while working at Fraunhofer FIT

show abstract

“…Such algorithms may or may not exploit ontological information, but this does not affect our central hypothesis. [14] also uses network structure to evaluate link quality, but in a very different way. The main intuition there is that an individual link in an ILN is more reliable when it leads to a greater information gain.…”

Section: Related Workmentioning

confidence: 99%

Network metrics for assessing the quality of entity resolution between multiple datasets1

Idrissou

¹

,

Harmelen

²

,

Besselaar

³

2020

View full text Add to dashboard Cite

Matching entities between datasets is a crucial step for combining multiple datasets on the semantic web. A rich literature exists on different approaches to this entity resolution problem. However, much less work has been done on how to assess the quality of such entity links once they have been generated. Evaluation methods for link quality are typically limited to either comparison with a ground truth dataset (which is often not available), manual work (which is cumbersome and prone to error), or crowd sourcing (which is not always feasible, especially if expert knowledge is required). Furthermore, the problem of link evaluation is greatly exacerbated for links between more than two datasets, because the number of possible links grows rapidly with the number of datasets. In this paper, we propose a method to estimate the quality of entity links between multiple datasets. We exploit the fact that the links between entities from multiple datasets form a network, and we show how simple metrics on this network can reliably predict their quality. We verify our results in a large experimental study using six datasets from the domain of science, technology and innovation studies, for which we created a gold standard. This gold standard, available online, is an additional contribution of this paper. In addition, we evaluate our metric on a recently published gold standard to confirm our findings.

show abstract

“…Information Gain. The work in [14] also uses network structure to evaluate link quality, but in a very different way. The main intuition there is that an individual link in an ILN is more reliable when it leads to a greater information gain.…”

Section: Related Workmentioning

confidence: 99%

Network Metrics for Assessing the Quality of Entity Resolution Between Multiple Datasets

Idrissou

¹

,

Harmelen

²

,

Besselaar

³

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Matching entities between datasets is a crucial step for combining multiple datasets on the semantic web. A rich literature exists on different approaches to this entity resolution problem. However, much less work has been done on how to assess the quality of such entity links once they have been generated. Evaluation methods for link quality are typically limited to either comparison with a ground truth dataset (which is often not available), manual work (which is cumbersome and prone to error), or crowd sourcing (which is not always feasible, especially if expert knowledge is required). Furthermore, the problem of link evaluation is greatly exacerbated for links between more than two datasets, because the number of possible links grows rapidly with the number of datasets. In this paper, we propose a method to estimate the quality of entity links between multiple datasets. We exploit the fact that the links between entities from multiple datasets form a network, and we show how simple metrics on this network can reliably predict their quality. We verify our results in a large experimental study using six datasets from the domain of science, technology and innovation studies, for which we created a gold standard. This gold standard, available online, is an additional contribution of this paper. In addition, we evaluate our metric on a recently published gold standard to confirm our findings.

show abstract

Methods for Intrinsic Evaluation of Links in the Web of Data

Cited by 8 publications

References 13 publications

Secure Evaluation of Knowledge Graph Merging Gain

Secure Evaluation of Knowledge Graph Merging Gain

Network metrics for assessing the quality of entity resolution between multiple datasets1

Network Metrics for Assessing the Quality of Entity Resolution Between Multiple Datasets

Contact Info

Product

Resources

About