Automatic distractor generation for multiple-choice English vocabulary questions

Susanti, Yuni; Tokunaga, Takenobu; Nishikawa, Hitoshi; Obari, Hiroyuki

doi:10.1186/s41039-018-0082-z

Cited by 29 publications

(17 citation statements)

References 15 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These tests can evaluate syntactic, semantic, or discourse knowledge, depending on the type of the word that is removed. Inspired by the approach of Susanti et al (2018), we generate incoherent variants of a document by replacing a randomly-selected word (noun or verb) in every sentence in the original text with an automatically-generated distractor. The distractors are chosen such that they fit in the sentence, but have a different meaning, e.g., we may replace sold for bought.…”

Section: Sentence Clozementioning

confidence: 99%

How coherent are neural models of coherence?

Pishdad¹,

Fancellu²,

Zhang³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Despite the recent advances in coherence modelling, most such models including state-of-the-art neural ones, are evaluated on either contrived proxy tasks such as the standard order discrimination benchmark, or tasks that require special expert annotation. Moreover, most evaluations are conducted on small newswire corpora. To address these shortcomings, in this paper we propose four generic evaluation tasks that draw on different aspects of coherence at both the lexical and document levels, and can be applied to any corpora. In designing these tasks, we aim at capturing coherence-specific properties, such as the correct use of discourse connectives, lexical cohesion, as well as the overall temporal and causal consistency among events and participants in a story. Importantly, our proposed tasks either rely on automatically-generated data, or data annotated for other purposes, hence alleviating the need for annotation specifically targeted to the task of coherence modelling. We perform experiments with several existing state-of-the-art neural models of coherence on these tasks, across large corpora from different domains, including newswire, dialogue, as well as narrative and instructional text. Our findings point to a strong need for revisiting the common practices in the development and evaluation of coherence models.

show abstract

Section: Sentence Clozementioning

confidence: 99%

How coherent are neural models of coherence?

Pishdad¹,

Fancellu²,

Zhang³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Distractors, which are the wrong answers set among the alternatives in a multiple‐choice test item, make the question an interesting and popular one. The distractors are similar enough to answer‐key and their purpose is to confuse the learner to give the correct answer . Normally, WordNet, domain ontologies or knowledge base is used to find similar or related words for generating distractors.…”

Section: Proposed Methodsmentioning

confidence: 99%

Automatic generation of fill‐in‐the‐blank question with corpus‐based distractors for e‐assessment to enhance learning

Das

Majumder

Phadikar

et al. 2019

Comp Applic In Engineering

View full text Add to dashboard Cite

Knowledge acquisition is the prime objective of a learner from an educational system and evaluating the learner's knowledge is the eventual goal of an examination process. This paper introduces a system which is able to produce fill‐in‐the‐blank questions to test the knowledge of a learner that he or she has accumulated after reading a course material. The question generation task is subdivided into three modules: sentence selection, answer‐key identification, and question formation along with distractor generation. The sentence is selected using a coarse‐grain part‐of‐speech tagset. The answer‐key is extracted by identifying topic‐word in the sentence and question is formed by omitting this topic‐word from the sentence. This paper also highlights an efficient corpus‐based distractor generation technique to produce multiple‐choice fill‐in‐the‐blank test items.

show abstract

“…Other approaches consider grammatical correctness, and introduce structural similarities in an ontology (Stasaski and Hearst, 2017), and syntactic similarities (Chen et al, 2006). When using broader context, bigram or n-gram co-occurrence (Susanti et al, 2018;Hill and Simha, 2016), context similarity (Pino et al, 2008), and context sensitive inference (Zesch and Melamud, 2014) have also been applied to distractor selection.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

2020

View full text Add to dashboard Cite

Readability assessment aims to automatically classify text by the level appropriate for learning readers. Traditional approaches to this task utilize a variety of linguistically motivated features paired with simple machine learning models. More recent methods have improved performance by discarding these features and utilizing deep learning models. However, it is unknown whether augmenting deep learning models with linguistically motivated features would improve performance further. This paper combines these two approaches with the goal of improving overall model performance and addressing this question. Evaluating on two large readability corpora, we find that, given sufficient training data, augmenting deep learning models with linguistically motivated features does not improve state-of-the-art performance. Our results provide preliminary evidence for the hypothesis that the state-of-theart deep learning models represent linguistic features of the text related to readability. Future research on the nature of representations formed in these models can shed light on the learned features and their relations to linguistically motivated ones hypothesized in traditional approaches.

show abstract

Automatic distractor generation for multiple-choice English vocabulary questions

Cited by 29 publications

References 15 publications

How coherent are neural models of coherence?

How coherent are neural models of coherence?

Automatic generation of fill‐in‐the‐blank question with corpus‐based distractors for e‐assessment to enhance learning

Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications

Contact Info

Product

Resources

About