The Qanary Ecosystem: Getting New Insights by Composing Question Answering Pipelines

Abstract

Abstract. The field of Question Answering (QA) is very multi-disciplinary as it requires expertise from a large number of areas such as natural language processing (NLP), artificial intelligence, machine learning, information retrieval, speech recognition and semantic technologies. In the past years a large number of QA systems were proposed using approaches from different fields and focusing on particular tasks in the QA process. Unfortunately, most of these systems cannot be easily reused, extended, and results cannot be easily reproduced since the systems are mostly implemented in a monolithic fashion, lack standardized interfaces and are often not open source or available as Web services. To address these issues we developed the knowledge-based Qanary methodology for choreographing QA pipelines distributed over the Web. Qanary employs the qa vocabulary as an exchange format for typical QA components. As a result, QA systems can be built using the Qanary methodology in a simpler, more flexible and standardized way while becoming knowledgedriven instead of being process-oriented. This paper presents the components and services that are integrated using the qa vocabulary and the Qanary methodology within the Qanary ecosystem. Moreover, we show how the Qanary ecosystem can be used to analyse QA processes to detect weaknesses and research gaps. We illustrate this by focusing on the Entity Linking (EL) task w.r.t. textual natural language input, which is a fundamental step in most QA processes. Additionally, we contribute the first EL benchmark for QA, as open source. Our main goal is to show how the research community can use Qanary to gain new insights into QA processes.

Core techniques of question answering systems over knowledge bases: a survey

“…We are aware of four frameworks that attempt to provide a reusable architecture for QA systems. QALL-ME (Ferrandez et al 2011), openQA (Marx et al 2014), OKBQA 16 and QANARY (Both et al 2016, Diefenbach, Singh, Both, Cherix, Lange & Auer 2017. Such integration would also allow to build and reuse services around QA systems like speech recognition modules and reusable front-ends (Diefenbach, Amjad, Both, Singh & Maret 2017).…”

Towards a question answering system over the Semantic Web

“…First, we integrated WDAqua-core1 into Qanary [5,12], a framework to integrate QA components. This way WDAqua-core1 can be accessed via RESTful interfaces for example to benchmark it via Gerbil for QA [45].…”

Trill: A Reusable Front-End for QA Systems

“…To achieve the modularity by making the front-end independent from the back-end of the QA system, we build Trill on top of a framework that provides a reusable architecture for QA systems. We are aware of four such architectures namely: QALL-ME [6], openQA [7], the Open KnowledgeBase and Question-Answering (OKBQA) challenge 7 and Qanary [9,1,4]. We choose to build Trill on top of the last mentioned framework, Qanary, since it allows the highest flexibility to integrate new QA components.…”