Qanary – The Fast Track to Creating a Question Answering System with Linked Data Technology

Singh, K. M.; Both, Andreas; Diefenbach, Dennis; Saedeeh, Shekarpour,; Cherix, Didier; Lange, Christoph

doi:10.1007/978-3-319-47602-5_36

Cited by 32 publications

(11 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These systems are: Currently, GERBIL QA supports the addition of 3 types of systems: (1) services implemented as Java- 15 https://uima.apache.org/ 1, we implemented the 4 systems which were available as webservice and returned Linked Data. We tested option (2) using the recent QA-NARY [35] framework. Option (3) was tested with the QALD-6 data and will be used for the 7th instantiation of the QALD challenge.…”

Section: Datasetsmentioning

confidence: 99%

Benchmarking question answering systems

Usbeck

Röder

Hoffmann

et al. 2019

View full text Add to dashboard Cite

The necessity of making the Semantic Web more accessible for lay users, alongside the uptake of interactive systems and smart assistants for the Web, have spawned a new generation of RDF-based question answering systems. However, fair evaluation of these systems remains a challenge due to the different type of answers that they provide. Hence, repeating current published experiments or even benchmarking on the same datasets remains a complex and time-consuming task. We present a novel online benchmarking platform for question answering (QA) that relies on the FAIR principles to support the fine-grained evaluation of question answering systems. We detail how the platform addresses the fair benchmarking platform of question answering systems through the rewriting of URIs and URLs. In addition, we implement different evaluation metrics, measures, datasets and pre-implemented systems as well as methods to work with novel formats for interactive and non-interactive benchmarking of question answering systems. Our analysis of current frameworks shows that most of the current frameworks are tailored towards particular datasets and challenges but do not provide generic models. In addition, while most frameworks perform well in the annotation of entities and properties, the generation of SPARQL queries from annotated text remains a challenge.

show abstract

Section: Datasetsmentioning

confidence: 99%

Benchmarking question answering systems

Usbeck

Röder

Hoffmann

et al. 2019

View full text Add to dashboard Cite

show abstract

“…We have designed and implemented GeoQA, the first question answering system for linked geospatial data. GeoQA is implemented using reusable components as part of the component-oriented Qanary question answering methodology [7,8] and its most recent implementation Frankestein [9].…”

Section: Introductionmentioning

confidence: 99%

Template-Based Question Answering over Linked Geospatial Data

Punjani

Singh

Both

et al. 2018

Proceedings of the 12th Workshop on Geographic Information Retrieval

View full text Add to dashboard Cite

Large amounts of geospatial data have been made available recently on the linked open data cloud and the portals of many national cartographic agencies (e.g., OpenStreetMap data, administrative geographies of various countries, or land cover/land use data sets). These datasets use various geospatial vocabularies and can be queried using SPARQL or its OGC-standardized extension GeoSPARQL. In this paper, we go beyond these approaches to offer a question-answering engine for natural language questions on top of linked geospatial data sources. Our system has been implemented as re-usable components of the Frankenstein question answering architecture. We give a detailed description of the system's architecture, its underlying algorithms, and its evaluation using a set of 201 natural language questions. The set of questions is offered to the research community as a gold standard dataset for the comparative evaluation of future geospatial question answering engines.

show abstract

“…Ask Wikidata 3 offers an easy way to post queries to Wikidata via a web-based interface, though the tool relies on manual disambiguation to understand a question. The WDAqua project 4 has produced a speech-based plug-in interface (Kumar et al, 2017) and the Qanary specification for QA systems (Singh et al, 2016). These tools follow the described steps of the QA pipeline, but do not facilitate the interactive instance-based evaluation that is the main aspect of this work.…”

Section: Introductionmentioning

confidence: 99%

Interactive Instance-based Evaluation of Knowledge Base Question Answering

Sorokin

Gurevych

2018

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

Most approaches to Knowledge Base Question Answering are based on semantic parsing. In this paper, we present a tool that aids in debugging of question answering systems that construct a structured semantic representation for the input question. Previous work has largely focused on building question answering interfaces or evaluation frameworks that unify multiple data sets. The primary objective of our system is to enable interactive debugging of model predictions on individual instances (questions) and to simplify manual error analysis. Our interactive interface helps researchers to understand the shortcomings of a particular model, qualitatively analyze the complete pipeline and compare different models. A set of sit-by sessions was used to validate our interface design.

show abstract

Qanary – The Fast Track to Creating a Question Answering System with Linked Data Technology

Cited by 32 publications

References 8 publications

Benchmarking question answering systems

Benchmarking question answering systems

Template-Based Question Answering over Linked Geospatial Data

Interactive Instance-based Evaluation of Knowledge Base Question Answering

Contact Info

Product

Resources

About