A Meta-Analysis and Systematic Review of the Effect of Chatbot Technology Use in Sustainable Education

Abstract

The development of artificial intelligence in recent years has promoted the use of chatbot technology in sustainable education. Many studies examined the effect of chatbots on learning outcomes. However, scant studies summarized the effectiveness of chatbots in education. The aim of the study is to investigate the effect of chatbot-assisted learning on various components and how different moderator variables influenced the effectiveness. This study, through a meta-analysis, reviewed 32 empirical studies with 2201 participants published between 2010 and 2022. The results showed that chatbot technology exerted a medium-to-high effect on overall learning outcomes regardless of moderator variables, i.e., intervention duration, chatbot roles, and learning content. In terms of learning components, chatbots could significantly improve explicit reasoning, learning achievement, knowledge retention, and learning interest despite negative findings in critical thinking, learning engagement, and motivation. Future research could expand chatbot research by including different learning components.

Exploring the dynamics of ChatGPT: Students and lecturers perspectives at an open distance e-learning university

“…The potential misuse of ChatGPT as a shortcut raises questions about equitable access. Deng andYu (2023), andJoshi et al (2023) caution against over-reliance on technology, and the findings here amplify this concern. This resonates with the CoI's emphasis on maintaining a balance between technology-driven engagement and authentic human interaction.…”

“…Globally conducted studies highlight the positive correlation between ChatGPT utilisation and heightened student engagement (Cotton et al, 2023;Kasneci et al, 2023). Deng and Yu (2023) demonstrated its effectiveness as a teaching assistant, resulting in increased motivation and engagement among online students. This highlights the potential for personalised feedback and assistance in the online learning context.…”

“…The 24/7 availability and personalised responses of ChatGPT can potentially bolster social interaction among students. However, the observed risk of over-reliance and passivity introduces a critical element that challenges the equilibrium between technologydriven engagement and authentic human interaction (Deng & Yu, 2023). Within the CoI framework, this raises questions about how technology influences social presence and whether it might lead to a potential imbalance.…”

“…Integrating this into the CoI framework highlights potential shifts in social and teaching presence. This raises vital questions about how technology-driven interactivity impacts authentic human interactions and personalised guidance, highlighting the delicate balance advocated by the CoI between technology-driven engagement and human presence in ODeL (Deng & Yu, 2023).…”

What ChatGPT means for universities: Perceptions of scholars and students

“…The utilization of AI-based chatbots in educational activities represents a significant domain for supporting student engagement and learning processes. Research has demonstrated that chatbot technologies can enhance student interaction and learning processes (D'Mello et al, 2014), enrich learning experiences by impacting student success in higher education (Winkler & Söllner, 2018), and potentially improve student motivation, engagement, and learning outcomes (Deng & Yu, 2023). However, it is not yet possible to assert a consensus among educators, specifically concerning ChatGPT.…”