Third Point of View Augmented Reality for Robot Intentions Visualization

Ruffaldi, Emanuele; Brizzi, Filippo; Tecchia, Franco; Bacinelli, Sandro

doi:10.1007/978-3-319-40621-3_35

Cited by 23 publications

(15 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Optical HMDs can overlay 3D graphics on top of the real world from the point of view of the human. This has been hypothesized to be a natural and transparent means of robot intent communication, for instance, with the overlay of future robot poses (Ruffaldi et al, 2016; Scassellati and Hayes, 2014). Hopefully such a system would reduce human–robot collaborative task time and produce fewer errors.…”

Section: Related Workmentioning

confidence: 99%

“…Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA Natural communication might be achieved when humans can see a robot's future motion in the real world from their own point of view, via a head-mounted display (HMD) (Ruffaldi et al, 2016;Scassellati and Hayes, 2014). This could increase safety and efficiency as the human no longer needs to divert their attention.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays

Rosen

Whitney

Phillips

et al. 2019

The International Journal of Robotics Research

View full text Add to dashboard Cite

Efficient motion intent communication is necessary for safe and collaborative work environments with co-located humans and robots. Humans efficiently communicate their motion intent to other humans through gestures, gaze, and other non-verbal cues, and can replan their motions in response. However, robots often have difficulty using these methods. Many existing methods for robot motion intent communication rely on 2D displays, which require the human to continually pause their work to check a visualization. We propose a mixed-reality head-mounted display (HMD) visualization of the intended robot motion over the wearer’s real-world view of the robot and its environment. In addition, our interface allows users to adjust the intended goal pose of the end effector using hand gestures. We describe its implementation, which connects a ROS-enabled robot to the HoloLens using ROS Reality, using MoveIt for motion planning, and using Unity to render the visualization. To evaluate the effectiveness of this system against a 2D display visualization and against no visualization, we asked 32 participants to label various arm trajectories as either colliding or non-colliding with blocks arranged on a table. We found a 15% increase in accuracy with a 38% decrease in the time it took to complete the task compared with the next best system. These results demonstrate that a mixed-reality HMD allows a human to determine where the robot is going to move more quickly and accurately than existing baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays

Rosen

Whitney

Phillips

et al. 2019

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…Several projects explored the idea of visualizing the robot's future intentions using AR interfaces. Virtual 3D arrows can be employed to display the future motion of the EE [10] or the movement of a virtual representation of an industrial manipulator can be visualized to improve the safety perception [45]. Further, projected AR interfaces can be used to visualize the objects that will be manipulated by the industrial robots.…”

Section: Related Work a Augmented Reality And Industrial Robotsmentioning

confidence: 99%

“…Thirty-four people, with ages that ranged from 19 to 30 years, have evaluated both modalities, alternating every time the A-NA, NA-A orders to limit learnability effects. Participants were all volunteers and they gave written informed consent in accordance with the Declaration of Helsinki 10 .…”

Section: Experimental Testsmentioning

confidence: 99%

See 1 more Smart Citation

An Augmented Reality System to Support Fault Visualization in Industrial Robotic Tasks

et al. 2019

View full text Add to dashboard Cite

The digitalization is transforming the very nature of factories, from automated systems to intelligent ones. In this process, industrial robots play a key role. Even if repeatability, precision and velocity of the industrial manipulators enable reaching considerable production levels, factories are required to face an increasingly competitive market, which requires being able to dynamically adapt to different situations and conditions. Hence, facilities are moving toward systems that rely on the collaboration between humans and machines. Human workers should understand the behavior of the robots, placing trust in them to properly collaborate. If a fault occurs on a manipulator, its movements are suddenly stopped for security reasons, thus workers may not be able to understand what happened to the robot. Therefore, the operators' stress and anxiety may increase, compromising the human-robot collaborative scenario. This work fits in this context and it proposes an adaptive Augmented Reality system to display industrial robot faults by means of the Microsoft HoloLens device. Starting from the methodology employed to identify which virtual metaphors best evoke robot faults, an adaptive modality is presented to dynamically display the metaphors in positions close to the fault location, always visible from the user and not occluded by the manipulator. A comparison with a non adaptive modality is proposed to assess the effectiveness of the adaptive solution. Results show that the adaptive modality allows users to recognize faults faster and with fewer movements than the non adaptive one, thus overcoming the limitation of the narrow field-of-view of the HoloLens device.

show abstract

Communicating Robot Arm Motion Intent Through Mixed Reality Head-Mounted Displays

Rosen

Whitney

Phillips

et al. 2019

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

Efficient motion intent communication is necessary for safe and collaborative work environments with collocated humans and robots. Humans efficiently communicate their motion intent to other humans through gestures, gaze, and social cues. However, robots often have difficulty efficiently communicating their motion intent to humans via these methods. Many existing methods for robot motion intent communication rely on 2D displays, which require the human to continually pause their work and check a visualization. We propose a mixed reality head-mounted display visualization of the proposed robot motion over the wearer's real-world view of the robot and its environment. To evaluate the effectiveness of this system against a 2D display visualization and against no visualization, we asked 32 participants to labeled different robot arm motions as either colliding or non-colliding with blocks on a table. We found a 16% increase in accuracy with a 62% decrease in the time it took to complete the task compared to the next best system. This demonstrates that a mixed-reality HMD allows a human to more quickly and accurately tell where the robot is going to move than the compared baselines.

show abstract

Third Point of View Augmented Reality for Robot Intentions Visualization

Cited by 23 publications

References 14 publications

Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays

Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays

An Augmented Reality System to Support Fault Visualization in Industrial Robotic Tasks

Communicating Robot Arm Motion Intent Through Mixed Reality Head-Mounted Displays

Contact Info

Product

Resources

About