The Virtual Tutor: Combining Conversational Agents with Learning Analytics to support Formative Assessment in Online Collaborative Learning

Abstract

The objective of this design science research project is to combine Learning Analytics data with a conversational agent communication interface, the Virtual Tutor, which is able to support formative assessment for educators and learners in online collaborative learning (OCL) environments. The main benefit for educators is providing user-adaptable Learning Analytics data requests to fit the information needs for formative assessment. Learners receive semiautomated feedback on their platform activity in form of reports, which shall trigger self-reflection processes. By extracting requirements from the potential users and deriving design principles, a conversational agent is implemented and evaluated in an online collaborative learning course. The results indicate that the Virtual Tutor reduces the task load of educators, supports formative assessment and gives scaffolded guidance to the learners by reflecting their performance, thus triggering self-reflection processes. This research provides a first step towards data supported (semi-) automated feedback systems for formative assessment in OCL courses.

(How) Can Machine Learning Support Teaching Staff in a Virtual Collaborative Learning Environment?

“…Since Artificial Intelligence in combination with learning analytics during VCL projects became a big topic in recent years, it is important to consider the existing related research (Salas-Pilco et al, 2022). To support the teaching staff's work there already are several concepts for conversational agents providing eTutors and students with statistics and information to improve formative assessment (Lenk & Schmidt, 2022). Furthermore, they also can be used to create automated feedback systems which notify students in case their contributions regarding the task lie below a certain limit (Porter & Grippa, 2020).…”

“…Based on the identified challenges we had to develop a concept for a software prototype that can support the challenges of eTutors. If we look at the research of Lenk and Schmidt (2022), we can see how important it is to present information to eTutors in a graphical and easily understandable manner. Based on this research and the challenges identified, we have established two main requirements:…”

“…In the context of this research VCL projects take place in settings where teaching staff (within our research scope, we were primarily focused on eTutors) assign tasks to the students, who then must organize their workflow amongst themselves to solve given problems. The eTutors are often responsible for multiple groups over longer periods of time which results in difficulties keeping an overview and providing every group with the necessary support and feedback (Lenk & Schmidt, 2022).…”