Augmented Reality User Interface for Mobile Robots With Manipulator Arms: Development, Testing, and Qualitative Analysis

Abstract

Augmented Reality (AR) combines real-world visual data with an overlay of computer graphics, enhancing the original video feed. AR can improve communication between robots and users via an intuitive spatial and visual dialogue, making it an attractive tool for teleoperated robot user interfaces (UIs). Many Unmanned Ground Vehicle (UGV) teleoperation tasks require both chassis navigation and manipulator arm control; however existing literature usually focuses on one task or the other, so there exists a gap in mobile robot UIs that take advantage of AR for both navigation and manipulation. This work describes the development and initial testing of a low-cost alpha-design UI for a teleoperated mobile manipulation task that uses AR technology to help the user feel more present in the robot’s environment. The results of the initial user tests are to be used to guide the future development of the user interface.

Registration errors in beacon-based navigation guidance systems: Influences on path efficiency and user reliance

“…There is some empirical evidence that perceptible registration error in landmark annotation can adversely impact users' trust in the reliability of augmented reality. Users explicitly note losing trust in augmented reality when landmarks are tagged with annotations that do not reliably register, in dynamic ways, with the directly perceived landmark (Vozar and Tilbury, 2012;Wither et al, 2011). For instance, users may lose trust when a beacon notes the location of the library, but is displaced by some degree of angular error from the library itself.…”

The Effects of Interface Views on Performing Aerial Telemanipulation Tasks Using Small UAVs

“…The Mission Specialist is the role responsible for a mission-specific task, while the Pilot maintains responsibility for navigation. The use of a dedicated Mission Specialist interface (or an equivalent, dedicated operator) for remote manipulation tasks has been studied in the literature for ground vehicles [7,10,11], underwater vehicles [12,13], and stationary manipulators [14,15]. In UAS field operations, however, there are a different set of design constraints compared to more "traditional" telemanipulation systems that use large or stationary control stations.…”

Review of Human–Machine Interfaces for Small Unmanned Systems With Robotic Manipulators

“…Dual joysticks (e.g., gamepad controllers) provide input if the slave manipulator requires control for both orientation and translation [24]. Joysticks and gamepad controllers have operated manipulators on ROVs [31], [32], UGVs [33], and simulated UAVs [34].…”

“…Control input devices are a critical link between operators and teleoperated systems in remote environments [53], [54]. The form factor and design of input devices influence intuitiveness and learning requirements, which subsequently affect operator performance [33], [34]. The sections below discuss how telemanipulation interfaces affect operator performance, and includes the following interface devices-physical hand controllers, workstations, and touch-based mobile devices.…”

“…Depending on the level and location of control within the manipulator, each design affords different types of interaction between the controller device and robot. For example, hand controllers that are geometrically similar and retain all robotic manipulator degrees of freedom are normally easier for users to learn because they intuitively map user input to output [9], [33]. A remote operator identifies their body and immediate environment with the remote vehicle and its environment; therefore, a geometrically similar device better enables the match between their own movements and the remote manipulator attached to the vehicle [55].…”