Non-parametric Class Completeness Estimators for Collaborative Knowledge Graphs—The Case of Wikidata

Abstract

Collaborative Knowledge Graph platforms allow humans and automated scripts to collaborate in creating, updating and interlinking entities and facts. To ensure both the completeness of the data as well as a uniform coverage of the different topics, it is crucial to identify underrepresented classes in the Knowledge Graph. In this paper, we tackle this problem by developing statistical techniques for class cardinality estimation in collaborative Knowledge Graph platforms. Our method is able to estimate the completeness of a class-as defined by a schema or ontology-hence can be used to answer questions such as "Does the knowledge base have a complete list of all {Beer Brands-Volcanos-Video Game Consoles}?" As a use-case, we focus on Wikidata, which poses unique challenges in terms of the size of its ontology, the number of users actively populating its graph, and its extremely dynamic nature. Our techniques are derived from species estimation and data-management methodologies, and are applied to the case of graphs and collaborative editing. In our empirical evaluation, we observe that i) the number and frequency of unique class instances drastically influence the performance of an estimator, ii) bursts of inserts cause some estimators to overestimate the true size of the class if they are not properly handled, and iii) one can effectively measure the convergence of a class towards its true size by considering the stability of an estimator against the number of available instances.

Wiki2Prop: A Multimodal Approach for Predicting Wikidata Properties from Wikipedia

“…For example, predicting entity types aims at recommending a type for new instances, Moon et al [22], propose to learn embeddings for entity types. Luggen et al [19] proposed to tackle the task of identifying incomplete classes in a knowledge graph by leveraging editor activity patterns. Another line of work focuses on link prediction or identifying the relationship between two input entities.…”

Predicting Document Coverage for Relation Extraction

“…1. Wikidata (GT wiki ): A popular KB providing data for most relations yet having coverage limitations (Galárraga et al, 2017;Luggen et al, 2019). For example, for Bill Gates, Microsoft and other popularly associated companies for the member-of relation are present, but niche entities like Honeywell are missing.…”

“…For question answering over knowledge bases, it is important to know whether a KB can be relied upon in terms of complete answer sets (Darari et al, 2013;Hopkinson et al, 2018;Arnaout et al, 2021). Current coverage estimation techniques for KBs do this analysis only post-hoc after the KB is fully constructed (Galárraga et al, 2017;Luggen et al, 2019), losing access to valuable information from extraction time.…”

Commonsense Knowledge in Wikidata

“…Studies of completeness of Wikidata Several papers study the completeness of Wikidata [4,2,16]. Luggen et al [16] provide an approach to estimate class completeness in knowledge graphs, and use Wikidata as a use case. They note that some classes in Wikidata, like Painting, are more complete than others, such as Mountain.…”