Knowledge Questions from Knowledge Graphs

Abstract

We address the novel problem of automatically generating quiz-style knowledge questions from a knowledge graph such as DBpedia. Questions of this kind have ample applications, for instance, to educate users about or to evaluate their knowledge in a specific domain. To solve the problem, we propose an end-to-end approach. The approach first selects a named entity from the knowledge graph as an answer. It then generates a structured triple-pattern query, which yields the answer as its sole result. If a multiplechoice question is desired, the approach selects alternative answer options. Finally, our approach uses a template-based method to verbalize the structured query and yield a natural language question. A key challenge is estimating how difficult the generated question is to human users. To do this, we make use of historical data from the Jeopardy! quiz show and a semantically annotated Web-scale document collection, engineer suitable features, and train a logistic regression classifier to predict question difficulty. Experiments demonstrate the viability of our overall approach.

A Systematic Review of Automatic Question Generation for Educational Purposes

“…the number of times distractors appear in a corpus is similar to the key) ( Kwankajornkiet et al 2016;Susanti et al 2015) selected distractors that are declared in a KB to be siblings of the key, which also implies some notion of similarity (siblings are assumed to be similar). Another approach that relies on structured knowledge sources is described in Seyler et al (2017). The authors used query relaxation, whereby queries used to generate question keys are relaxed to provide distractors that share some of the key features.…”

“…Eight of these studies focus on the difficulty of questions belonging to a particular domain, such as mathematical word problems (Wang and Su 2016;Khodeir et al 2018), geometry questions (Singhal et al 2016), vocabulary questions (Susanti et al 2017a), reading comprehension questions (Gao et al 2018), DFA problems (Shenoy et al 2016), code-tracing questions (Thomas et al 2019), and medical case-based questions Kurdi et al 2019). The remaining six focus on controlling the difficulty of non-domain-specific questions (Lin et al 2015;Alsubait et al 2016;Kurdi et al 2017;Faizan and Lohmann 2018;Faizan et al 2017;Seyler et al 2017;Kumar 2015a, 2017a;Vinu et al 2016;Kumar 2017b, 2015b). Table 6 shows the different features proposed for controlling question difficulty in the aforementioned studies.…”

“…Difficulty control was validated by checking agreement between predicted difficulty and expert prediction in Vinu and Kumar (2015b), Alsubait et al (2016), Seyler et al (2017), Khodeir et al (2018), and Leo et al (2019), by checking agreement between predicted difficulty and student performance in Alsubait et al (2016), Susanti et al (2017a), Lin et al (2015), Wang and Su (2016), Leo et al (2019), and Thomas et al (2019), by employing automatic solvers in Gao et al (2018), or by asking experts to complete a survey after using the tool (Singhal et al 2016). Expert reviews and mock exams are equally represented (seven studies each).…”

“…Aspects of verbalisation that have been considered are pronoun substitutions (i.e. replacing pronouns by their antecedents) (Huang and He 2016), selection of a suitable auxiliary verb (Mazidi and Nielsen 2015), determiner selection (Zhang and VanLehn 2016), and representation of semantic entities (Vinu and Kumar 2015b;Seyler et al 2017) (see below for more on this).…”

Evaluating the quality of the ontology-based auto-generated questions

“…Different qualitative and quantitative analyses were carried out to evaluate questions auto-generated from domain ontologies (Alsubait et al 2014;Vinu and Kumar 2017;Seyler et al 2016;Susanti et al 2017). Papasalouros et al (2017;2011) autogenerated multiple choice questions (MCQs) from the Eupalineio Tunnel ontology, which is a domain ontology about the ancient Greek history.…”