Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks

Abstract

This paper addresses the task of (complex) conversational question answering over a knowledge graph. For this task, we propose LASAGNE (muLti-task semAntic parSing with trAnsformer and Graph atteNtion nEtworks). It is the first approach, which employs a transformer architecture extended with Graph Attention Networks for multi-task neural semantic parsing. LASAGNE uses a transformer model for generating the base logical forms, while the Graph Attention model is used to exploit correlations between (entity) types and predicates to produce node representations. LASAGNE also includes a novel entity recognition module which detects, links, and ranks all relevant entities in the question context. We evaluate LASAGNE on a standard dataset for complex sequential question answering, on which it outperforms existing baseline averages on all question types. Specifically, we show that LASAGNE improves the F1-score on eight out of ten question types; in some cases, the increase in F1-score is more than 20% compared to the state of the art.

VOGUE: Answer Verbalization through Multi-Task Learning

“…Model Configuration For simplicity, to represent the logical forms, we employ the same grammar as in (Kacupaj et al, 2021b). Our approach can be used with any other grammar or even directly with SPARQL queries.…”

“…For the logical forms, we employ a grammar that can be used to capture the entire context of the question with the minimum number of actions. We prefer not to reinvent the wheel, and therefore we adopted the grammar from existing stateof-the-art question answering systems (Kacupaj et al, 2021b;Plepi et al, 2021). However, we do not employ all the actions from these works; Table 7 illustrates the complete grammar with all the defined actions that we used for all three answer verbalization datasets.…”

“…In recent years, publicly available knowledge graphs (KG) (e.g., DBpedia (Lehmann et al, 2015), Wikidata (Vrandečić and Krötzsch, 2014)) have been broadly adopted as a source of knowledge in several tasks such as entity linking, relation extraction, and question answering (Kacupaj et al, 2021b). Question answering (QA) over knowledge graphs, in particular, is an essential task that maps a user's utterance to a query over a KG to retrieve the correct answer (Singh et al, 2018).…”

“…This work argues that leveraging content from both sources allows the model for better convergence and provides new, improved results. Furthermore, we complement our hypothesis with multi-task learning paradigms, since multi-task learning has been quite efficient for different system architectures (Cipolla et al, 2018), including question answering systems (Plepi et al, 2021;Kacupaj et al, 2021b). Our framework can receive two (e.g., question & query) or even one (e.g., question) input.…”

A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspective

“…(Chen Jr and Bunescu, 2019;Jia and Liang, 2016;Yin and Neubig, 2017a;Rabinovich et al, 2017;Ling et al, 2016;Sun et al, 2019a)), and transformer-based architectures (e.g. (Kacupaj et al, 2021;Ferraro and Suominen, 2020;Shen et al, 2019;Sun et al, 2019b;Gemmell et al, 2020;Kusupati and Ailavarapu)). In the late 1970's, Hendrix et al (1978) pioneer the task of interfacing with databases using natural language.…”