Augmented Reality for Robotics: A Review

Abstract

Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.

An Application-Based Review of Haptics Technology

“…Haptic rendering refers to determining the haptic force, and visual rendering enables us to visually display the interaction of a virtual object [59]. A VR or AR system is normally constructed with head-mounted displays such as goggles, accelerometers, and loudspeakers, and is based on these elements to generate a computer-based environment for an operator to interact with the environment [60,61]. Besides, audio and video, skin-based sensory systems are underdeveloped in VR and AR technology [59,60,62], which were proved to provide an enhanced experience in several applications such as entertainment and medicine.…”

Multimodal Mixed Reality Impact on a Hand Guiding Task with a Holographic Cobot

“…Human-robot interaction (HRI) has made profilic use of augmented reality, as seen in recent review publications, such as in [14], which identifies a roadmap for the field. In the same way, in [15], a review of advanced robot programming approaches is presented where the authors explain the current augmented reality-based programming approaches.…”

Engineering Reliable Interactions in the Reality-Artificiality Continuum

“…Just one month later the Toyota Research Institute released a video showing how they use VR to allow human trainers to teach robots in-home tasks 7 . The just published paper by Makhataeva and Varol [40] surveys about 100 AR applications in medical robotics, robot control and planning, human-robot interaction and robot swarms from 2015 and 2019. Recent experiments demonstrate the value in using modern VR/AR technologies to mediate human-robot interactions [51] and boost robots acceptance by their users.…”