Studying the Effect of Tutor Learning Using a Teachable Agent that Asks the Student Tutor for Explanations

Matsuda, Noboru; Cohen, William W.; Koedinger, Kenneth R.; Keiser, Victoria; Raizada, Rohan; Yarzebinski, Evelyn; Watson, Shayna; Stylianides, Gabriel J.

doi:10.1109/digitel.2012.12

Cited by 23 publications

(28 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the learning sciences, we see a parallel line of research about learning by mentoring [4,20]. Mentoring also involves providing assessment and instructions to others.…”

Section: Learning By Reviewingmentioning

confidence: 97%

“…Theory from organization science (e.g., [36]) and learning science (e.g., [20]) would suggest that reviewing others' work can offer an opportunity to make a meaningful impact, induce a sense of responsibility, and improve motivation to master a task domain. This paper explores ancillary benefits-such as learning and performance increases-that peer reviewing can provide to the reviewers, not just to workers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reviewing versus doing

Zhu

Dow

Kraut

et al. 2014

Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work &Amp; Social Computing

View full text Add to dashboard Cite

In modern crowdsourcing markets, requesters face the challenge of training and managing large transient workforces. Requesters can hire peer workers to review others' work, but the value may be marginal, especially if the reviewers lack requisite knowledge. Our research explores if and how workers learn and improve their performance in a task domain by serving as peer reviewers. Further, we investigate whether peer reviewing may be more effective in teams where the reviewers can reach consensus through discussion. An online between-subjects experiment compares the tradeoffs of reviewing versus producing work using three different organization strategies: working individually, working as an interactive team, and aggregating individuals into nominal groups. The results show that workers who review others' work perform better on subsequent tasks than workers who just produce. We also find that interactive reviewer teams outperform individual reviewers on all quality measures. However, aggregating individual reviewers into nominal groups produces better quality assessments than interactive teams, except in task domains where discussion helps overcome individual misconceptions.

show abstract

“…In the learning sciences, we see a parallel line of research about learning by mentoring [4,20]. Mentoring also involves providing assessment and instructions to others.…”

Section: Learning By Reviewingmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Reviewing versus doing

Zhu

Dow

Kraut

et al. 2014

Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work &Amp; Social Computing

View full text Add to dashboard Cite

show abstract

“…Yeah, let's do it! Some technically attractive efforts such as free-form explanation feedback (Popescu et al 2003), collaborative peer tutoring (Walker et al 2009), hand-written equation entry (Anthony et al 2005), and teachable agents (Matsuda et al 2012), have shown promise, but have not always produced clear demonstrations of improving student learning outcomes above and beyond those achieved by the existing tutor. Some approaches to supporting student metacognition and motivation, such as help-seeking (Roll et al 2014), sense-making on errors (Mathan and Koedinger 2003), or reducing gaming the system (Baker et al 2006) have been demonstrated to improve outcomes, but have not been incorporated, perhaps in part because of the cost of technical development.…”

Section: Q) What Was Particularly Challenging In Undertaking This Study?mentioning

confidence: 99%

An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City”

Koedinger

Aleven

2015

Int J Artif Intell Educ

Self Cite

207

198

View full text Add to dashboard Cite

Our 1997 article in IJAIED reported on a study that showed that a new algebra curriculum with an embedded intelligent tutoring system (the Algebra Cognitive Tutor) dramatically enhanced high-school students' learning. The main motivation for the study was to demonstrate that intelligent tutors that have cognitive science research embedded in them could have real impact in schools. This study was one of the first large-scale classroom evaluations of the integrated use of an Intelligent Tutoring System (ITS) in high schools. A core challenge was figuring out how to embed this new technology into a curriculum and into the existing social context of schools. A key element of the study design was to include multiple kinds of assessments, including standardized test items and items measuring complex problem solving and use of representations. The results were powerful: BOn average the 470 students in experimental classes outperformed students in comparison classes by 15 % on standardized tests and 100 % on tests targeting the [course] objectives.^We suggested that the study was evidence Bthat laboratory tutoring systems can be scaled up and made to work, both technically and pedagogically, in real and unforgiving settings like urban high schools.^Since this study, many more classroom studies comparing instruction that includes an ITS against business as usual have been conducted, often showing advantages for the ITS-enhanced curricula. More rigorous randomized field trials are now more commonplace, but the approach of using multiple assessments in large-scale randomized field trials has not caught on. Cognitive task analysis will remain fundamental to the success of ITSs. A key remaining question for ITS is to find out how they can be used most effectively to support open-ended problem solving, either online or offline. Given all the recent excitement around Massive Open Online Courses (MOOCs), it is interesting to note that our field of

show abstract

“…A more detailed analysis of the 3 rd grade students' results showed that the game playing students' higher gains mainly derived from problems concerning conceptual understanding of the base-10 system (Pareto et al 2012). The version of SimStudent that enables learning-byteaching (Matsuda et al 2010) was used in a three-day classroom experimental study with 160 high school students (Matsuda et al 2012). The study compared two versions of the system: with and without a self-explanation module.…”

Section: Related Work: Learning By Teaching Systems and Teachable Agentsmentioning

confidence: 99%

“…Hence, the agent acts as "an ideal learner" and poses numerous deep, explanatory questions prompting the student-tutor to reflect on and explain her playing behaviour. Recently, Matsuda et al (2012) also added explanatory questions to their TA to prompt for self-explanation. The purpose of the teachable agent dialogue is three-fold; to challenge students to reflect on why they play as they do; to support transfer of tacit game playing knowledge to articulate language; and to provide a role model of questioning to imitate.…”

Section: Our Approachmentioning

confidence: 99%

A Teachable Agent Game Engaging Primary School Children to Learn Arithmetic Concepts and Reasoning

Pareto

2014

Int J Artif Intell Educ

View full text Add to dashboard Cite

In this paper we will describe a learning environment designed to foster conceptual understanding and reasoning in mathematics among younger school children. The learning environment consists of 48 2-player game variants based on a graphical model of arithmetic where the mathematical content is intrinsically interwoven with the game idea. The environment also features teachable agents, which are computer programs that can be taught and behave according to their knowledge. Thus, the environment provides both learning-by-doing (playing the game) and learning-byteaching (teaching the agent to play). It differs from other learning-by-teaching systems 1) by targeting basic mathematics and primary grade students; 2) by using teachable agents as an extension to educational games in order to leverage engagement, reflection and learning; and 3) by using an agent-driven question dialogue to challenge students' mathematical thinking, to role-model learner behaviour and to transfer game knowledge to out-of-game mathematics. The teachable agent game is described and evaluated in an authentic classroom study enrolling 443 students from 22 classes in 9 schools. Students range from 2 nd to 6 th grade of mainstream classes and 7 th to 8 th grade for students with difficulties in mathematics. Part of the study was designed as a quasiexperimental study with controls; part was designed to examine students' change in mental models of arithmetic before and after game play. All students took pre-and post mathematics tests. The 314 playing students used the game and taught their agents during regular math-classes for three months, whereas the control classes attended standard instruction and took the tests. A questionnaire was distributed at the end of the study to investigate students' perceptions and performances of the agent-tutoring task. Results show that 1) there is a significant learning gain for playing students compared to controls, 2) the learning environment can engage children in advanced mathematical thinking in early education, 3) young primary students can act as successful tutors. Thus, we conclude that teachable agents in educational games can help achieve deeper levels of learning that transfer outside the game. This idea combines the motivational power of games with the reflective power of a teachable agent asking thoughprovoking, deep questions on the learning material during game play.

show abstract

Studying the Effect of Tutor Learning Using a Teachable Agent that Asks the Student Tutor for Explanations

Cited by 23 publications

References 14 publications

Reviewing versus doing

Reviewing versus doing

An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City”

A Teachable Agent Game Engaging Primary School Children to Learn Arithmetic Concepts and Reasoning

Contact Info

Product

Resources

About