Automatic Generation of English Vocabulary Tests

Susanti, Yuni; Iida, Ryu; Tokunaga, Takenobu

doi:10.5220/0005437200770087

Cited by 26 publications

(37 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results in their study showed that their proposed method ensured the validity of the items and produced fewer problematic distractors than the baseline method, being also comparable with that of human-made distractors. Their findings coincided to a similar earlier study, where Susanti et al (2015) attempted to construct English vocabulary automatic tests in which human evaluation indicated that 45% of the responses from English teachers mistakenly judged the automatically generated questions to be human-generated questions. Finally, Ha and Yaneva (2018)…”

Section: B) Option Weighting Practicessupporting

confidence: 87%

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Ypsilandis¹,

Mouti²

2019

JLE

View full text Add to dashboard Cite

One among the main concerns of language testers in the design and implementation of tests is selecting the method of scoring for the tool used to perform the evaluation. This attribute indirectly reveals the tester’s ethical beliefs and personal stance on testing pedagogy. This is another study challenging the typical 1-0 method of scoring in Multiple Choice Tests (MCT) and implements, for experimental purposes, a simple polychotomous partial-credit scoring system on official tests administered for the National Foreign Language Exam System in Greece (NFLES-Gr). The study comes in support of earlier findings on the subject by the same authors in analogous smaller-scale studies. The MCT items chosen were completed by a total of 1,922 subjects in different levels of the NFLES-Gr test for Italian as an L2 in Greece. Results clearly indicate that the tested scoring procedure provides refined insights into students’ interlanguage levels, enhances sensitivity in scoring procedures, and may provide significant differences for testees found to be close to the pass/non-pass borderline without jeopardizing test reliability.

show abstract

Section: B) Option Weighting Practicessupporting

confidence: 87%

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Ypsilandis¹,

Mouti²

2019

JLE

View full text Add to dashboard Cite

show abstract

“…Still other approaches to content selection are more specific and are informed by the type of question to be generated. For example, the purpose of the study reported in Susanti et al (2015) is to generate "closest-in-meaning vocabulary questions" 9 which involve selecting a text snippet from the Internet that contains the target word, while making sure that the word has the same sense in both the input and retrieved sentences. To this end, the retrieved text was scored on the basis of metrics such as the number of query words that appear in the text.…”

Section: Generation Tasksmentioning

confidence: 99%

“…Among these are selection of distractors based on word frequency (i.e. the number of times distractors appear in a corpus is similar to the key) ( Kwankajornkiet et al 2016;Susanti et al 2015) selected distractors that are declared in a KB to be siblings of the key, which also implies some notion of similarity (siblings are assumed to be similar). Another approach that relies on structured knowledge sources is described in Seyler et al (2017).…”

Section: Generation Tasksmentioning

confidence: 99%

“…Wang et al (2018) claimed that these metrics can be used as initial, inexpensive, large-scale indicators of the fluency and relevancy of questions. Other researchers investigated whether machine-generated questions are indistinguishable from human-authored questions by mixing both types and asking experts about the source of each question Susanti et al 2015;Khodeir et al 2018). Some researchers evaluated their approaches by investigating the ability of the approach to assemble human-authored distractors.…”

Section: Types Of Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Automatic Question Generation for Educational Purposes

Kurdi

Leo

Parsia

et al. 2019

Int J Artif Intell Educ

239

111

View full text Add to dashboard Cite

While exam-style questions are a fundamental educational tool serving a variety of purposes, manual construction of questions is a complex process that requires training, experience, and resources. This, in turn, hinders and slows down the use of educational activities (e.g. providing practice questions) and new advances (e.g. adaptive testing) that require a large pool of questions. To reduce the expenses associated with manual construction of questions and to satisfy the need for a continuous supply of new questions, automatic question generation (AQG) techniques were introduced. This review extends a previous review on AQG literature that has been published up to late 2014. It includes 93 papers that were between 2015 and early 2019 and tackle the automatic generation of questions for educational purposes. The aims of this review are to: provide an overview of the AQG community and its activities, summarise the current trends and advances in AQG, highlight the changes that the area has undergone in the recent years, and suggest areas for improvement and future opportunities for AQG. Similar to what was found previously, there is little focus in the current literature on generating questions of controlled difficulty, enriching question forms and structures, automating template construction, improving presentation, and generating feedback. Our findings also suggest the need to further improve experimental reporting, harmonise evaluation metrics, and investigate other evaluation methods that are more feasible.

show abstract

“…Most approaches require the distractor and the target word to have the same part-of-speech (POS) and similar level of difficulty, often approximated by word frequency (Coniam, 1997;Shei, 2001;Brown et al, 2005). They must also be semantically close, which can be quantified with semantic distance in WordNet (Lin et al, 2007;Pino et al, 2008;Chen et al, 2015;Susanti et al, 2015), thesauri (Sumita et al, 2005;Smith et al, 2010), ontologies (Karamanis et al, 2006;Ding and Gu, 2010), or handcrafted rules (Chen et al, 2006). Another approach generates distractors that are semantically similar to the target word in some sense, but not in the particular sense in the carrier sentence (Zesch and Melamud, 2014).…”

Section: Previous Workmentioning

confidence: 99%

Distractor Generation for Chinese Fill-in-the-blank Items

Jiang¹,

Lee²

2017

Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

This paper reports the first study on automatic generation of distractors for fill-inthe-blank items for learning Chinese vocabulary. We investigate the quality of distractors generated by a number of criteria, including part-of-speech, difficulty level, spelling, word co-occurrence and semantic similarity. Evaluations show that a semantic similarity measure, based on the word2vec model, yields distractors that are significantly more plausible than those generated by baseline methods.

show abstract

Automatic Generation of English Vocabulary Tests

Cited by 26 publications

References 8 publications

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

A Systematic Review of Automatic Question Generation for Educational Purposes

Distractor Generation for Chinese Fill-in-the-blank Items

Contact Info

Product

Resources

About