Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval

Kulshreshtha, Devang; Belfer, Robert; Serban, Iulian Vlad; Reddy, Siva

doi:10.18653/v1/2021.emnlp-main.566

Cited by 6 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen et al [5] create a large-scale Educational QG dataset from KhanAcademy and TED-Ed data sources as a learning and assessment tools for students. Kulshreshtha et al [17] also release a QG dataset comprising of data-science questions to promote research in domain adaptation. Unlike our questions, the questions in Chen et al [5], Kulshreshtha et al [17] are static and not personalized to the student.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…BART is a Transformer autoencoder pre-trained to reconstruct text from noisy text inputs. For QG, it learns a conditional probablity distribution P(q|r) to generate question q from reference solution r. We experiment with two pre-trained checkpoints -a) original BART-base checkpoint provided by authors and b) BART model trained on 50K MLQuestions dataset using back-training algorithm [17]. The latter model is able to generate good-quality questions for data science domain which is also our domain of interest in Korbit ITS.…”

Section: Few-shot Question Generation (Qg) Modelmentioning

confidence: 99%

“…Kulshreshtha et al [17] also release a QG dataset comprising of data-science questions to promote research in domain adaptation. Unlike our questions, the questions in Chen et al [5], Kulshreshtha et al [17] are static and not personalized to the student. A recent work by Srivastava and Goodman [33] generates personalized questions according to the student's level by proposing a difficulty-controllable QG model.…”

Section: Related Workmentioning

confidence: 99%

“…We find BART outperforms T5 by 4 BLEU1 points, showing that BART is better suited for conditional generation. Also pre-training on MLQuestions dataset[17] increases BLEU1 by 1.5 absolute points.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

Kulshreshtha

Shayan²,

Belfer³

et al. 2022

Pais 2022

Self Cite

View full text Add to dashboard Cite

Existing work on generating hints in Intelligent Tutoring Systems (ITS) focuses mostly on manual and non-personalized feedback. In this work, we explore automatically generated questions as personalized feedback in an ITS. Our personalized feedback can pinpoint correct and incorrect or missing phrases in student answers as well as guide them towards correct answer by asking a question in natural language. Our approach combines cause–effect analysis to break down student answers using text similarity-based NLP Transformer models to identify correct and incorrect or missing parts. We train a few-shot Neural Question Generation and Question Re-ranking models to show questions addressing components missing in the student’s answers which steers students towards the correct answer. Our model vastly outperforms both simple and strong baselines in terms of student learning gains by 45% and 35% respectively when tested in a real dialogue-based ITS. Finally, we show that our personalized corrective feedback system has the potential to improve Generative Question Answering systems.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Few-shot Question Generation (Qg) Modelmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

Kulshreshtha

Shayan²,

Belfer³

et al. 2022

Pais 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, many recent works Kulshreshtha et al, 2021) find that the domain generalization ability of dense retrieval models is weak. Inspired by , we introduce two out-of-domain testing sets from the medical domain, including cMedQA and cCOVID-News † as the separate testing sets (see Section 2.4).…”

Section: Introductionmentioning

confidence: 99%

DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine

Qiu¹,

Li²,

Qu³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we present DuReader retrieval , a large-scale Chinese dataset for passage retrieval. DuReader retrieval contains more than 90K queries and over 8M unique passages from Baidu search. To ensure the quality of our benchmark and address the shortcomings in other existing datasets, we (1) reduce the false negatives in development and testing sets by pooling the results from multiple retrievers with human annotations, (2) and remove the semantically similar questions between training with development and testing sets. We further introduce two extra out-of-domain testing sets for benchmarking the domain generalization capability. Our experiment results demonstrate that DuReader retrieval is challenging and there is still plenty of room for the community to improve, e.g. the generalization across domains, salient phrase and syntax mismatch between query and paragraph and robustness. DuReader retrieval will be publicly available at https: //github.com/baidu/DuReader/ tree/master/DuReader-Retrieval

show abstract

Review on Neural Question Generation for Education Purposes

Al Faraby,

Adiwijaya,

Romadhony

2023

Int J Artif Intell Educ

View full text Add to dashboard Cite

Questioning plays a vital role in education, directing knowledge construction and assessing students’ understanding. However, creating high-level questions requires significant creativity and effort. Automatic question generation is expected to facilitate the generation of not only fluent and relevant but also educationally valuable questions. While rule-based methods are intuitive for short inputs, they struggle with longer and more complex inputs. Neural question generation (NQG) has shown better results in this regard. This review summarizes the advancements in NQG between 2016 and early 2022. The focus is on the development of NQG for educational purposes, including challenges and research opportunities. We found that although NQG can generate fluent and relevant factoid-type questions, few studies focus on education. Specifically, there is limited literature using context in the form of multi-paragraphs, which due to the input limitation of the current deep learning techniques, require key content identification. The desirable key content should be important to specific topics or learning objectives and be able to generate certain types of questions. A further research opportunity is controllable NQG systems, which can be customized by taking into account factors like difficulty level, desired answer type, and other individualized needs. Equally important, the results of our review also suggest that it is necessary to create datasets specific to the question generation tasks with annotations that support better learning for neural-based methods.

show abstract

Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval

Cited by 6 publications

References 41 publications

Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

Few-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems

DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine

Review on Neural Question Generation for Education Purposes

Contact Info

Product

Resources

About