A Model-Free Affective Reinforcement Learning Approach to Personalization of an Autonomous Social Robot Companion for Early Literacy Education

Abstract

Personalized education technologies capable of delivering adaptive interventions could play an important role in addressing the needs of diverse young learners at a critical time of school readiness. We present an innovative personalized social robot learning companion system that utilizes children’s verbal and nonverbal affective cues to modulate their engagement and maximize their long-term learning gains. We propose an affective reinforcement learning approach to train a personalized policy for each student during an educational activity where a child and a robot tell stories to each other. Using the personalized policy, the robot selects stories that are optimized for each child’s engagement and linguistic skill progression. We recruited 67 bilingual and English language learners between the ages of 4–6 years old to participate in a between-subjects study to evaluate our system. Over a three-month deployment in schools, a unique storytelling policy was trained to deliver a personalized story curriculum for each child in the Personalized group. We compared their engagement and learning outcomes to a Non-personalized group with a fixed curriculum robot, and a baseline group that had no robot intervention. In the Personalization condition, our results show that the affective policy successfully personalized to each child to boost their engagement and outcomes with respect to learning and retaining more target words as well as using more target syntax structures as compared to children in the other groups.

Assessing Children's Perceptions and Acceptance of a Social Robot

“…The robot led children in a conversation about a set of pictures (depicting holidays, school activities, a park, and children's movies). It told children a story, and then asked children to retell the story, similar to the activities used in [35,49]. We administered the PST and SAQ after children completed the robot interaction.…”

“…Social robots and related technologies are increasingly used with children in longitudinal contexts, such as education, healthcare, and therapy [6,23,40,43,49,59,60]. Because the goals of child-robot interactions in these areas-namely learning, behavior change, and improving health-necessarily take time, the interactions children have are thus longer-term, with repeated encounters over weeks or months.…”

“…Some recent research has begun examining children's long-term interactions with social robots, especially in education [1,25,30,33,40,49,56,57]. In these interactions, robots are often situated as relational agents, i.e., as agents that actively try to build and maintain long-term, social-emotional relationships [9].…”

Impact of Interaction Context on the Student Affect-Learning Relationship in Child-Robot Interaction

“…Prior research has often assumed a fixed, uni-directional relationship between an affective state and learning outcomes across an entire learning interaction -without examining the modulatory effect of interaction context nor incorporating this interdependence into agent behavior models [15,26,37]. For example, Woolf et al [37] implemented a simple rule-based affect extension in their virtual tutoring system to deal with students' affective states.…”

“…Guidelines that capture the affect-learning relationship in different contexts would be very useful in the design of affective pedagogical robots. However, prior work in affective pedagogical robots seldom interpret students' affective displays with respect to different social learning paradigms or in different interaction contexts [15,26,31,33]. Instead, a single model or policy is often applied across different contexts.…”