Deep Discourse Analysis for Generating Personalized Feedback in Intelligent Tutor Systems

Abstract

We explore creating automated, personalized feedback in an intelligent tutoring system (ITS). Our goal is to pinpoint correct and incorrect concepts in student answers in order to achieve better student learning gains. Although automatic methods for providing personalized feedback exist, they do not explicitly inform students about which concepts in their answers are correct or incorrect. Our approach involves decomposing students answers using neural discourse segmentation and classification techniques. This decomposition yields a relational graph over all discourse units covered by the reference solutions and student answers. We use this inferred relational graph structure and a neural classifier to match student answers with reference solutions and generate personalized feedback. Although the process is completely automated and data-driven, the personalized feedback generated is highly contextual, domain-aware and effectively targets each student's misconceptions and knowledge gaps. We test our method in a dialogue-based ITS and demonstrate that our approach results in high-quality feedback and significantly improved student learning gains.

Comparative Analysis of Generative Pre-Trained Transformers Responses to Coastal Ecosystem Questions: Implications for GenAI in Environmental Education

“…The quality of classroom discourse is vital for effective learning, and emerging technologies such as GenAI offer new ways to enhance educational experiences [10]; [11]; [12]. GenAI, in particular, has the potential to support productive student discussions, provide personalized learning experiences, and encourage creativity across different domains [13]; [14]. However, the capacity of GenAI to obscure the distinction between original and AI-generated content raises ethical issues, requiring careful consideration of its educational use [15].…”

Catalyzing Equity in STEM Teams: Harnessing Generative AI for Inclusion and Diversity

“…Intelligent tutoring systems (ITS) employ advanced AI models to support students’ learning paths, tracking progress, and enabling tutors to provide targeted guidance ( Rovira et al, 2017 ; Neumann et al, 2021 ). Dialogue-based ITSs use large language models for personalized, context-aware feedback ( Grenander et al, 2021 ). In team learning, AI agents sensitive to psychological and problem states to aid in collaborative problem-solving ( Graesser et al, 2018a ).…”

DIRECT: Toward Dialogue-Based Reading Comprehension Tutoring

“…Deep learning-based ITSs for generating feedback and questions have achieved promising results. Reference [11] deployed an ITS for generating personalized feedback. To provide instruction to students about concepts that were misunderstood, they analyzed the relationship between the answer and ground knowledge using natural language processing techniques such as segmentation and sematic parsing.…”

“…Moreover, they could engage in small talk, which has been shown to be an efficient tutoring technique for encouraging student engagement [9]. To address the diversity of tutoring strategies and difficulty of understanding and responding educationally to students' utterances, data-driven approaches with natural language processing techniques have been introduced for ITSs [10], [11]. However, there are few public datasets handling the diverse tutoring strategies in the discourse, despite their importance to the development and evaluation of data-driven models.…”