Improving Unsupervised Question Answering via Summarization-Informed Question Generation

Abstract

Question Generation (QG) is the task of generating a plausible question for a given <passage, answer> pair. Template-based QG uses linguistically-informed heuristics to transform declarative sentences into interrogatives, whereas supervised QG uses existing Question Answering (QA) datasets to train a system to generate a question given a passage and an answer. A disadvantage of the heuristic approach is that the generated questions are heavily tied to their declarative counterparts. A disadvantage of the supervised approach is that they are heavily tied to the domain/language of the QA dataset used as training data. In order to overcome these shortcomings, we propose an unsupervised QG method which uses questions generated heuristically from summaries as a source of training data for a QG system. We make use of freely available news summary data, transforming declarative summary sentences into appropriate questions using heuristics informed by dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model. We extrinsically evaluate our approach using unsupervised QA: our QG model is used to generate synthetic QA pairs for training a QA model. Experimental results show that, trained with only 20k English Wikipedia-based synthetic QA pairs, the QA model substantially outperforms previous unsupervised models on three in-domain datasets (SQuAD1.1, Natural Questions, TriviaQA) and three out-of-domain datasets (NewsQA, BioASQ, DuoRC), demonstrating the transferability of the approach.

A Feasibility Study of Answer-Unaware Question Generation for Education

“…Future work should seek to further explore the relationship between summarization and QG. Work done concurrently to ours by Lyu et al (2021) already has promising results in this direction, showing that training a QG model on synthetic data from summarized text improves performance on downstream QA.…”

All You May Need for VQA are Image Captions

“…It is explored as a standalone task (Heilman and Smith, 2009;Nema et al, 2019), as a pre-training task for language models (Narayan et al, 2020) and as a component in solutions for other textual tasks, such as question answering Puri et al, 2020), information retrieval (Mass et al, 2020;Gaur et al, 2021) and generation evaluation (Durmus et al, 2020;Honovich et al, 2021). There are two main directions to QG: template-based (Heilman and Smith, 2009;Lyu et al, 2021;Dhole and Manning, 2020) and neural-based, with the latter achieving state-of-the-art results Narayan et al, 2020).…”

Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs

“…Similarly, paraphrasing also demonstrates students' content comprehension skills (Haynes and Fillmer, 1984) since it requires the ability of conveying the same semantic meaning with different languages. However, both tasks are rarely studied for question generation (Lyu et al, 2021). Simplification, on the other hand, has demonstrated its values in language learning and reading comprehension (Tweissi, 1998;Inui et al, 2003;Petersen and Ostendorf, 2007;Rets and Rogaten, 2021), and is often treated as a preprocessing step to convert complex sentences into simpler versions before creating questions (Majumder and Saha, 2015;Patra and Saha, 2019).…”