Visualizing Robot Intent for Object Handovers with Augmented Reality

Newbury, Rhys; Cosgun, Akansel; Crowley-Davis, Tysha; Chan, Wesley P.; Drummond, Tom; Croft, Elizabeth A.

doi:10.48550/arxiv.2103.04055

Cited by 9 publications

(8 citation statements)

References 15 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For such handovers, the robot would need to keep the target object in-sight as much as possible [33], and a faster gripper might be needed. It may also be useful to utilize a projection method such as augmented reality to make the robot's intent accessible to the users [34].…”

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%

On-The-Go Robot-to-Human Handovers with a Mobile Manipulator

He¹,

Simini²,

Chan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Existing approaches to direct robot-to-human handovers are typically implemented on fixed-base robot arms, or on mobile manipulators that come to a full stop before performing the handover. We propose "on-the-go" handovers which permit a moving mobile manipulator to hand over an object to a human without stopping. The on-the-go handover motion is generated with a reactive controller that allows simultaneous control of the base and the arm. In a user study, human receivers subjectively assessed on-the-go handovers to be more efficient, predictable, natural, better timed and safer than handovers that implemented a "stop-and-deliver" behavior.

show abstract

Section: Conclusion Limitations and Future Workmentioning

confidence: 99%

On-The-Go Robot-to-Human Handovers with a Mobile Manipulator

He¹,

Simini²,

Chan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A user study was conducted to test the effect of robot and motion intent visualization in AR. The methodology is inspired by our previous work on human-robot collaboration [29] and AR [8].…”

Section: User Study Designmentioning

confidence: 99%

“…Robots that move in predictable ways arguably improve the safety of the human-robot interaction in both of those aspects. Methods for generating predictable robot motion has previously used gaze [5], [6], light projection [7], plan visualization [8] or implicit robot motions [9], [10], however, these approaches assume that the robot is visible to the human. When the robot is not visible to the humans in the environment, it would not be possible to know where the robot is and where it is moving towards, or even if there is a robot in the occluded areas.…”

Section: Introductionmentioning

confidence: 99%

Seeing Thru Walls: Visualizing Mobile Robots in Augmented Reality

Gu,

Cosgun,

Chan

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present an approach for visualizing mobile robots through an Augmented Reality headset when there is no line-of-sight visibility between the robot and the human. Three elements are visualized in Augmented Reality: 1) Robot's 3D model to indicate its position, 2) An arrow emanating from the robot to indicate its planned movement direction, and 3) A 2D grid to represent the ground plane. We conduct a user study with 18 participants, in which each participant are asked to retrieve objects, one at a time, from stations at the two sides of a T-junction at the end of a hallway where a mobile robot is roaming. The results show that visualizations improved the perceived safety and efficiency of the task and led to participants being more comfortable with the robot within their personal spaces. Furthermore, visualizing the motion intent in addition to the robot model was found to be more effective than visualizing the robot model alone. The proposed system can improve the safety of automated warehouses by increasing the visibility and predictability of robots.

show abstract

“…In order to convey detailed and attention grabbing signals about robot inference we turn to augmented reality and haptic feedback. Augmented reality head-mounted displays have previously been used to facilitate human-robot interaction [15], [16]. Most relevant here are [17]- [20], where the authors leverage augmented reality to convey the robot's internal state, visualize where the robot plans to go, and establish bidirectional communication with the human.…”

Section: Related Workmentioning

confidence: 99%

Communicating Inferred Goals with Passive Augmented Reality and Active Haptic Feedback

Mullen¹,

Mosier²,

Chakrabarti³

et al. 2021

Preprint

View full text Add to dashboard Cite

Robots learn as they interact with humans. Consider a human teleoperating an assistive robot arm: as the human guides and corrects the arm's motion, the robot gathers information about the human's desired task. But how does the human know what their robot has inferred? Today's approaches often focus on conveying intent: for instance, using legible motions or gestures to indicate what the robot is planning. However, closing the loop on robot inference requires more than just revealing the robot's current policy: the robot should also display the alternatives it thinks are likely, and prompt the human teacher when additional guidance is necessary. In this paper we propose a multimodal approach for communicating robot inference that combines both passive and active feedback. Specifically, we leverage information-rich augmented reality to passively visualize what the robot has inferred, and attention-grabbing haptic wristbands to actively prompt and direct the human's teaching. We apply our system to shared autonomy tasks where the robot must infer the human's goal in real-time. Within this context, we integrate passive and active modalities into a single algorithmic framework that determines when and which type of feedback to provide. Combining both passive and active feedback experimentally outperforms single modality baselines; during an in-person user study, we demonstrate that our integrated approach increases how efficiently humans teach the robot while simultaneously decreasing the amount of time humans spend interacting with the robot. Videos here: https://youtu.be/swq_u4iIP-g

show abstract

Visualizing Robot Intent for Object Handovers with Augmented Reality

Cited by 9 publications

References 15 publications

On-The-Go Robot-to-Human Handovers with a Mobile Manipulator

On-The-Go Robot-to-Human Handovers with a Mobile Manipulator

Seeing Thru Walls: Visualizing Mobile Robots in Augmented Reality

Communicating Inferred Goals with Passive Augmented Reality and Active Haptic Feedback

Contact Info

Product

Resources

About