Report on the First Knowledge Graph Reasoning Challenge 2018

Abstract

A new challenge for knowledge graph reasoning started in 2018. Deep learning has promoted the application of artificial intelligence (AI) techniques to a wide variety of social problems. Accordingly, being able to explain the reason for an AI decision is becoming important to ensure the secure and safe use of AI techniques. Thus, we, the Special Interest Group on Semantic Web and Ontology of the Japanese Society for AI, organized a challenge calling for techniques that reason and/or estimate which characters are criminals while providing a reasonable explanation based on an open knowledge graph of a well-known Sherlock Holmes mystery story. This paper presents a summary report of the first challenge held in 2018, including the knowledge graph construction, the techniques proposed for reasoning and/or estimation, the evaluation metrics, and the results. The first prize went to an approach that formalized the problem as a constraint satisfaction problem and solved it using a lightweight formal method; the second prize went to an approach that used SPARQL and rules; the best resource prize went to a submission that constructed word embedding of characters from all sentences of Sherlock Holmes novels; and the best idea prize went to a discussion multi-agents model. We conclude this paper with the plans and issues for the next challenge in 2019.

RDF-star2Vec: RDF-star Graph Embeddings for Data Mining

“…Given that existing hyper-relational KG datasets such as WikiPeople, JF17K, and WD50K lack complex structures, they are unsuitable for evaluating the proposed method. Thus, we provide a dataset KGRC-RDF-star that is available for benchmarking the embeddings of complex RDF-star graphs by converting the KGRC-RDF 6 [15], [31] to RDF-star data, which is provided for International Knowledge Graph Reasoning Challenge (IKGRC). 7 The original KGRC-RDF is a set of eight KGs built based on Sherlock Holmes mystery stories and has been published as an RDF dataset for benchmarking XAI systems that can provide reasons for its decisions.…”

A User Interface for Sense-making of the Reasoning Process while Interacting with Robots

“…In contrast, in the data visualization community, many studies and interfaces proposed for sense-making. Especially, the knowledge graph is seen as a tool for enhancing the transparency and explainability of the intelligent system [2,7,15], which has been applied for many domains, such as visual debugging [12] or recommendation system [8].…”

Building Narrative Structures from Knowledge Graphs

“…More recent research on knowledge graphs emphasises the need to have more event-centric structures to facilitate the analysis of sequence of events. A few event-centric knowledge graphs were built from existing generic knowledge graphs [21], whereas others were built from text in a domain like the news [49] or a novel [31]. Such structures also acknowledge the fundamental need to integrate the temporal dimension of events, and temporal and dynamic knowledge are currently being more investigated.…”

“…There has been recent significant effort to build more specific, commonsense knowledge graphs. Such efforts comprise areas like generic commonsense [8,52], actions [31,42], social interactions [53] or psychology [55].…”

“…Finally, some work focuses on how to formally represent narratives and to extract them from inputs like texts or knowledge bases. For text, [4] inspects evolving stories in news articles, [13] proposes a benchmark for extracting temporal and causal relations from text, and [31] proposes a graph-based reasoning challenge to find the criminal in a Sherlock Holmes novel. [5] surveys methods to extract processes from text in business process management and emphasises process elements identified and evaluations performed.…”