Preliminary Evaluations of a Dialogue-Based Digital Tutor

Ventura, Marco; Chang, Maria Hsia; Foltz, Peter W.; Mukhi, Nirmal; Yarbro, Jessica; Salverda, Anne Pier; Behrens, John T.; Ahn, Jae‐wook; Ma, Tengfei; Dhamecha, Tejas I.; Marvaniya, Smit; Watson, Patrick; D'Helon, Cassius; Tejwani, Ravi; Afzal, Shazia

doi:10.1007/978-3-319-93846-2_90

Cited by 14 publications

(11 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An 8 th grader with a question about the Krebs Cycle needs different types of information than an undergraduate Biology major, for example. However, despite research into the possibility of automated question answering in education, there has been little effort to scale these systems, with considerably more energy going into tutor-led tutorial dialogue systems (Wolfe et al, 2013;Ventura et al, 2018). Building such a system is non-trivial for several reasons, first and foremost the complexity that arises from handling unforeseen queries that represent considerable variability in the use of human language.…”

Section: Introductionmentioning

confidence: 99%

Curio SmartChat : A system for Natural Language Question Answering for Self-Paced K-12 Learning

Raamadhurai¹,

Baker

Poduval³

2019

Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

During learning, students often have questions which they would benefit from responses to in real time. In class, a student can ask a question to a teacher. During homework, or even in class if the student is shy, it can be more difficult to receive a rapid response. In this work, we introduce Curio SmartChat, an automated question answering system for middle school Science topics. Our system has now been used by around 20,000 students who have so far asked over 100,000 questions. We present data on the challenge created by students' grammatical errors and spelling mistakes, and discuss our system's approach and degree of effectiveness at disambiguating questions that the system is initially unsure about. We also discuss the prevalence of student "small talk" not related to science topics, the pluses and minuses of this behavior, and how a system should respond to these conversational acts. We conclude with discussions and point to directions for potential future work.

show abstract

Section: Introductionmentioning

confidence: 99%

Curio SmartChat : A system for Natural Language Question Answering for Self-Paced K-12 Learning

Raamadhurai¹,

Baker

Poduval³

2019

Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

show abstract

“…Many ITS incorporate explicit student models and consider the development of a personalized curriculum and personalized feedback (Albacete et al, 2019;Chi et al, 2011;Lin et al, 2013;Munshi & Biswas, 2019;Rus et al, 2014a, b). In this respect, dialogue-based ITS have been shown to be some of the most promising tools for learning (Ahn et al, 2018;Graesser et al 2001, Nye et al, 2014Ventura et al, 2018), as they simulate the familiar learning environment of student-tutor interaction, which helps improve student confidence and motivation and leads to a better learning experience. In particular, dialogue-based ITS mimic the familiar student-tutor interaction setting by asking students questions and presenting them with problem-solving exercises, while also providing students with the opportunity to pose their own questions, request hints and explanations, and engage in other types of communication with the tutor.…”

Section: Intelligent Tutoring Systemsmentioning

confidence: 99%

Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

Kochmar

Belfer³

et al. 2021

Int J Artif Intell Educ

View full text Add to dashboard Cite

Intelligent tutoring systems (ITS) have been shown to be highly effective at promoting learning as compared to other computer-based instructional approaches. However, many ITS rely heavily on expert design and hand-crafted rules. This makes them difficult to build and transfer across domains and limits their potential efficacy. In this paper, we investigate how feedback in a large-scale ITS can be automatically generated in a data-driven way, and more specifically how personalization of feedback can lead to improvements in student performance outcomes. First, in this paper we propose a machine learning approach to generate personalized feedback in an automated way, which takes individual needs of students into account, while alleviating the need of expert intervention and design of hand-crafted rules. We leverage state-of-the-art machine learning and natural language processing techniques to provide students with personalized feedback using hints and Wikipedia-based explanations. Second, we demonstrate that personalized feedback leads to improved success rates at solving exercises in practice: our personalized feedback model is used in , a large-scale dialogue-based ITS with around 20,000 students launched in 2019. We present the results of experiments with students and show that the automated, data-driven, personalized feedback leads to a significant overall improvement of 22.95% in student performance outcomes and substantial improvements in the subjective evaluation of the feedback.

show abstract

“…The Korbit ITS is related to the work on dialogue-based ITS, such as the pioneering AutoTutor and the newer IBM Watson Tutor [1,6,7,13,19]. Although Korbit is highly constrained compared to existing dialogue-based ITS, a major innovation of Korbit lies in its modular, scalable design.…”

Section: The Korbit Itsmentioning

confidence: 99%

A Large-Scale, Open-Domain, Mixed-Interface Dialogue-Based ITS for STEM

Serban¹,

Gupta²,

Kochmar

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We present Korbit, a large-scale, open-domain, mixed-interface, dialogue-based intelligent tutoring system (ITS). Korbit uses machine learning, natural language processing and reinforcement learning to provide interactive, personalized learning online. Korbit has been designed to easily scale to thousands of subjects, by automating, standardizing and simplifying the content creation process. Unlike other ITS, a teacher can develop new learning modules for Korbit in a matter of hours. To facilitate learning across a wide range of STEM subjects, Korbit uses a mixed-interface, which includes videos, interactive dialogue-based exercises, question-answering, conceptual diagrams, mathematical exercises and gamification elements. Korbit has been built to scale to millions of students, by utilizing a state-of-the-art cloud-based micro-service architecture. Korbit launched its first course in 2019 and has over 7, 000 students have enrolled. Although Korbit was designed to be open-domain and highly scalable, A/B testing experiments with real-world students demonstrate that both student learning outcomes and student motivation are substantially improved compared to typical online courses.

show abstract

Preliminary Evaluations of a Dialogue-Based Digital Tutor

Cited by 14 publications

References 1 publication

Curio SmartChat : A system for Natural Language Question Answering for Self-Paced K-12 Learning

Curio SmartChat : A system for Natural Language Question Answering for Self-Paced K-12 Learning

Automated Data-Driven Generation of Personalized Pedagogical Interventions in Intelligent Tutoring Systems

A Large-Scale, Open-Domain, Mixed-Interface Dialogue-Based ITS for STEM

Contact Info

Product

Resources

About