Haptic and visual augmented reality interface for programming welding robots

Abstract

It is a challenging task for operators to program a remote robot for welding manipulation depending only on the visual information from the remote site. This paper proposes an intuitive user interface for programming welding robots remotely using augmented reality (AR) with haptic feedback. The proposed system uses a depth camera to reconstruct the surfaces of workpieces. A haptic input device is used to allow users to define welding paths along these surfaces. An AR user interface is developed to allow users to visualize and adjust the orientation of the welding torch. Compared with the traditional robotic welding path programming methods which rely on prior CAD models or contact between the robot end-effector and the workpiece, this proposed approach allows for fast and intuitive remote robotic welding path programming without prior knowledge of CAD models of the workpieces. The experimental results show that the proposed approach is a user-friendly interface and can assist users in obtaining an accurate welding path.

Augmented Reality for Robotics: A Review

“…During the experiments, the robot was able to modify its intended motion following AR instructions comprised of glyphs and paths. An experimental setup containing an AR-supported vision system and haptic feedback technology which was used for the remote control of a welding robot is presented in [77]. Liu et al [78] integrated an AR interface with Temporal And-Or Graph (T-AOG) algorithm to provide a robot programmer with information on the robot's hidden states (e.g., latent forces during interaction).…”

An Education Application for Teaching Robot Arm Manipulator Concepts Using Augmented Reality

“…AR has significant growth potential in advanced manufacturing developments; for example, Ni et al [12] created the haptic robot for programming welding paths. e main advantages of this prototype were a user-friendly interface and the possibility to improve this work through seam tracking sensors.…”

Teleoperación de un brazo robot Kinova MICO2 a través de un dispositivo Omni Bundle

“…Se ha utilizado teleoperación también en robots de exploración espacial, como los de la NASA en expediciones a Marte [1]. En otros estudios se ha utilizado realidad aumentada, conjuntamente con dispositivos hápticos PHANToM para programar robots de soldadura [5]. Mucho más relacionado con esta investigación se ha utilizado este dispositivo háptico con realimentación de fuerza en un sistema para operación láser [8].…”

“…Se ha utilizado teleoperación también en robots de exploración espacial, como los de la NASA en expediciones a Marte [1]. En otros estudios se ha utilizado realidad aumentada, conjuntamente con dispositivos hápticos PHANToM para programar robots de soldadura [5]. Mucho más relacionado con esta investigación se ha utilizado este dispositivo háptico con realimentación de fuerza en un sistema para operación láser [8].La propuesta de este estudio es utilizar el instrumento háptico Omni Bundle como dispositivo maestro para teleoperar un brazo robótico Kinova MICO2 en entornos cotidianos, con realimentación de esfuerzos, para dotar de sensibilidad del entorno al operador.…”