Generative Agents to support students learning progress

Abstract

Ongoing assessments in a course are crucial for tracking student performance and progress. However, generating and evaluating tests for each lesson and student can be time-consuming. Existing models for generating and evaluating question-answer pairs have had limited success. In recent years, large language models (LLMs) have become available as a service, offering more intelligent answering and evaluation capabilities. This research aims to leverage LLMs for generating questions, model answers, and evaluations while providing valuable feedback to students and decentralizing the dependency on faculty.We finetune existing LLMs and employ prompt engineering to direct the model toward specific tasks using different generative agents. One agent generates questions, another generates answers, and the third takes the human answers to evaluate and ensure the quality. Human evaluation is conducted through focus group analysis, and student progress and faculty feedback are tracked. Results demonstrate the potential of LLMs to provide automatic feedback and learning progress tracking for both students and faculty.In conclusion, this paper demonstrates the versatility of LLMs for various learning tasks, including question generation, model answer generation, and evaluation, all while providing personalized feedback to students. By identifying and addressing knowledge gaps, LLMs can support continuous evaluation and help students improve their understanding before semester exams. Furthermore, knowledge gaps from students identified by the agent can be highlighted and addressed through additional classes or support materials, potentially generated by the same model, leading to a more personalized learning experience.