Question Generation for Question Answering

Abstract

This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data. The contribution of the paper is 2-fold: First, two types of question generation approaches are proposed, one is a retrieval-based method using convolution neural network (CNN), the other is a generation-based method using recurrent neural network (RNN); Second, we show how to leverage the generated questions to improve existing question answering systems. We evaluate our question generation method for the answer sentence selection task on three benchmark datasets, including SQuAD, MS MARCO, and WikiQA. Experimental results show that, by using generated questions as an extra signal, significant QA improvement can be achieved.

Neural Duplicate Question Detection without Labeled Training Data

“…While WS-TB is related to the approaches mentioned before, DQG is is also related to question generation (QG). Most of the previous work in QG is in the context of reading comprehension (e.g., Du et al, 2017;Subramanian et al, 2018;Zhao et al, 2018;Du and Cardie, 2018) or QG for question answering (Duan et al, 2017). They substantially differ from our approach because they generate questions based on specific answer spans, while DQG generates a new title from a question's body that can be used as a question duplicate.…”

Open-Domain Neural Conversational Agents: The Step Towards Artificial General Intelligence

“…Several research have utilized question generation as a tool to improve the efficacy of question answering models [29], [30], [26]. Authors in [30] simultaneously train the model by alternating input data between question answering (QA) and question generation (QG), both in the same model.…”

“…This training framework successfully shown that by exploiting the "duality" of QA and QG improves both QA and QG. Another joint approach for QA and QG is presented in [29]. Authors uses approach presented in [30] to prove hypothesis that good question generation can improve QA performance.…”

“…Authors uses approach presented in [30] to prove hypothesis that good question generation can improve QA performance. Authors in [29] leverages convolutional neural network (CNN) and Recurrent Neural Network (RNN) for question generation in order to cover both question generation approaches; retrieval based and generation based. Authors demonstrated that the question generation method successfully improve existing question answering systems.…”

“…Whereas [26] demonstrated the "duality" of both QG and QA models are able to improve both tasks, authors in [29] manage to significantly improve QA using a good question generation approach and feed QA system www.ijacsa.thesai.org Despite the accomplishments of these state of the art research, these conversational agents are lacking in several aspects which hinder these agents from creating conversational experiences that are indistinguishable from humans, which is the key in creating a system that is able to pass the Turing Test. Aspects that are lacking in current state of the art conversational agents are:…”