“I Didn’t Understand, I´m Really Not Very Smart”—How Design of a Digital Tutee’s Self-Efficacy Affects Conversation and Student Behavior in a Digital Math Game

Abstract

How should a pedagogical agent in educational software be designed to support student learning? This question is complex seeing as there are many types of pedagogical agents and design features, and the effect on different student groups can vary. In this paper we explore the effects of designing a pedagogical agent’s self-efficacy in order to see what effects this has on students´ interaction with it. We have analyzed chat logs from an educational math game incorporating an agent, which acts as a digital tutee. The tutee expresses high or low self-efficacy through feedback given in the chat. This has been performed in relation to the students own self-efficacy. Our previous results indicated that it is more beneficial to design a digital tutee with low self-efficacy than one with high self-efficacy. In this paper, these results are further explored and explained in terms of an increase in the protégé effect and a reverse role modelling effect, whereby the students encourage digital tutees with low self-efficacy. However, there are indications of potential drawbacks that should be further investigated. Some students expressed frustration with the digital tutee with low self-efficacy. A future direction could be to look at more adaptive agents that change their self-efficacy over time as they learn.

The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots

“…The former can be conducted quantitatively, as in e.g., Xiao et al (2020), or qualitatively, as in Wang and Nakatsu (2013). Often, log analyses have been used as a measurement tool along with questionnaires and interviews in experiments, like in Tärning and Silvervarg (2019), or as a supplementary method in mixed method research, like in Xu et al (2017).…”

“…focused on chatbots pertaining to the customer service and help-desk domain (e.g., Ashktorab et al, 2019); 14.5% of the papers (N=12) investigated chatbots that are used in healthcare and well-being, three thereof are specifically addressed to mental healthcare (e.g., Fitzpatrick et al, 2017); 7.2% of the articles (N=6) examined pedagogical-educational chatbot (e.g., Tärning & Silvervarg, 2019); 3.6% of the papers (N=3) explored chatbots aimed at providing suggestions (e.g., Jin et al, 2019); 2.4% of the papers (N=2) examined chatbots that are thought to be used in the workplace supporting productivity (e.g., Kimani et al, 2019), while one paper (1.2% of the papers) focused on a Human Resource (HR) chatbot (Liao et al, 2018); two papers dealt with search chatbots (e.g., Avula et al, 2018); one paper focused on a chatbot that is meant to interact during art exhibitions (Candello et al, 2019). This shows how chatbots are versatile and are pervading many aspects of our everyday life (Table 5).…”

“…By and large, users' negative emotions emerging during interaction seem to be the main culprit for abandoning the conversation, whereas positive emotions not only play a major role in preventing communication breakdowns but may also improve the outcomes of a technology intervention. Tärning and Silvervarg (2019), for instance, studied fourth grade students' reactions to a pedagogical agent aimed at supporting learning outcomes, pointing out that responses to the digital tutees' feedback, competence, and attitude were emotionally loaded: students with low self-efficacy who showed a positive disposition toward the chatbot, e.g., expressing happiness regarding its abilities, performed better in learning tasks. By contrast, Wang and Nakatsu (2013) conducted a laboratory experiment with 64 subjects and found that when the language used by a chatbot expresses emotional states, the chatbot draws users' cognitive attention to the social aspect of their interaction partner, producing increased co-presence.…”

Trends of Game-Based Learning in Mathematics Education: A Systematic Review

“…The results obtained for the third sub-problem of the research show that the conducted studies are mainly aimed at determining the effect of teaching mathematics with games on mathematical achievement (Kokandy, 2021;Lee & Choi, 2020;McIntosh, 2018;Stanton, 2017;Tärning & Silvervarg, 2019;Watson-Huggins, 2018). This is followed by studies in which opinions about the use of games in teaching mathematics are discussed (Bragg, 2007;Deng, Wu, Chen & Peng, 2020;Doğan & Sönmez, 2019;Machaba, 2019).…”

Galleryist Anna: Chatbot as an Educational Resource

“…Ze studie Smutného a Schreiberové (2020) jasně vyplývá, že v současném vzdělávání jde o téma zásadní, populární, často zkoušené, ale současně spojené s poměrně omezenou reflexí na úrovni pedagogických teorií. Dominantní roli v odborné literatuře, zaměřené na vzdělávání pomocí chatbotů, hrají případové studie nebo reflexe zkušenosti (Herrmann-Werner et al, 2021;Stuij et al, 2021, Tärning & Silvervarg, 2019), ovšem jen s omezenou ambicí poskytnout širší didaktické kotvení nebo pedagogickou analýzu.…”