Using Socially Expressive Mixed Reality Arms for Enhancing Low-Expressivity Robots

Abstract

Expressivity-the use of multiple modalities to convey internal state and intent of a robot-is critical for interaction. Yet, due to cost, safety, and other constraints, many robots lack high degrees of physical expressivity. This paper explores using mixed reality to enhance a robot with limited expressivity by adding virtual arms that extend the robot's expressiveness. The arms, capable of a range of non-physically-constrained gestures, were evaluated in a between-subject study (n = 34) where participants engaged in a mixed reality mathematics task with a socially assistive robot. The study results indicate that the virtual arms added a higher degree of perceived emotion, helpfulness, and physical presence to the robot. Users who reported a higher perceived physical presence also found the robot to have a higher degree of social presence, ease of use, usefulness, and had a positive attitude toward using the robot with mixed reality. The results also demonstrate the users' ability to distinguish the virtual gestures' valence and intent.

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction: A Survey and Virtual Design Element Taxonomy

“…Body Extensions add virtual parts to a robot without changing its underlying form such as an aerial robot is still recognizable as a UAV even if a virtual arm is added to its chassis, which would not be the case if virtual imagery were overlaid on the aerial robot to make it look like a floating robotic eye [90] (see Figure 3-H). Extensions do not necessarily need to be traditional robot parts and might instead appear as human heads, animal limbs, or even imagined parts like magical wings [12,30] (see Figure 3-F).…”

“…Callouts are visualizations that communicate where a user should focus their attention. These VDEs use visualizations to attract attention to an object or location, such using a virtual arrow to show where a robot heard a sound or pointing at an object a user should look at [30] (see Figure 3-F).…”

“…Task Instructions enable humans and robots to effectively instruct and guide each other by communicating next steps in collaborative tasks. These instructions can take various forms such as explicit instructions written in text or more abstract instructions that inform a user what to do to accomplish a task such as using an Robot Extension Morphological Alteration VDE to add virtual arms to a robot that point at an object for a user to interact with [26,30] (see Figures 5-E).…”

Which One? Choosing Favorite Robot After Different Styles of Storytelling and Robots’ Conversation

“…Humans develop an emotional connection with robots that move during their interaction ( Lambert et al, 2019 ). Moreover, humans report higher perceived physical presence, helpfulness, emotion, and positive attitude toward a robot with limited expressivity added to its virtual arms ( Groechel et al, 2019 ). A friendly attitude is also important as in the human–machine interaction, a personalized experience with an intelligent device makes people feel more comfortable when they perceive the device/machine as trustworthy ( Karat, Bloom, and Karat 2004 ).…”

Exploring Mixed Reality Robot Communication Under Different types of Mental Workload

“…Williams et al [63] divides these new forms of non-egocentric visual gestures into allocentric visualizations that can be displayed in teammates' augmented reality head-mounted displays, and perspective-free visualizations that can be projected onto the ground. Recent work in this space has focused on allocentric gestures such as circles and arrows drawn over target objects [57,58], as well as ego-sensitive allocentric gestures such as virtual arms [17,18]. Williams et al [58], for example [see also 57], demonstrate that (non-ego-sensitive) allocentric virtual gestures, at least when tested in a simulated video-based experiment, have the potential to increase communication accuracy and efficiency, and, when paired with complex referring expressions, are viewed as more effective and likable than purely linguistic communication.…”