Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

Abstract

Virtual and augmented reality is increasingly prevalent in industrial applications, such as remote control of industrial machinery, due to recent advances in head-mounted display technologies and low-latency communications via 5G. However, the influence of augmentations and camera placement-based viewing positions on operator performance in telepresence systems remains unknown. In this paper, we investigate the joint effects of depth-aiding augmentations and viewing positions on the quality of experience for operators in augmented telepresence systems. A study was conducted with 27 non-expert participants using a real-time augmented telepresence system to perform a remote-controlled navigation and positioning task, with varied depth-aiding augmentations and viewing positions. The resulting quality of experience was analyzed via Likert opinion scales, task performance measurements, and simulator sickness evaluation. Results suggest that reducing the reliance on stereoscopic depth perception via camera placement has a significant benefit to operator performance and quality of experience. Conversely, the depth-aiding augmentations can partly mitigate the negative effects of inferior viewing positions. However the viewing-position based monoscopic and stereoscopic depth cues tend to dominate over cues based on augmentations. There is also a discrepancy between the participants' subjective opinions on augmentation helpfulness, and its observed effects on positioning task performance.

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

“…Participants can rate the sequences using a controller or touchpad of the HMD [50], avoiding removing the goggles. Additionally, it can be recorded on a paper [30], verbally [35], [36], [51], [52], or online using a web application [53].…”

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

“…In other non-entertainment contexts, augmented remote operation appears to benefit the operators in terms of their Quality of Experience (QoE) and task accomplishment, as demonstrated in [6,8,9,10,11]. The augmentation covered by our system is a relatively simple view manipulation, meant to explore the technical feasibility more than the specific operator experience.…”

“…1a shows an example of a remote system in a mine, where operators are shown direct video feeds from on-machinery cameras, under various connection methods (5G, wired ethernet) and video compression levels. However, going beyond the direct presentation of camera views, there is potential in using AR, view synthesis, and range sensing from Time-of-Flight (ToF) sensors such as Light Detection And Ranging (lidar) to further enhance the operator awareness of the on-site environment [7,8]. The benefits of augmented and indirect views have been investigated in other contexts such as underwater robot operation [9], forestry [6], hazard exploration [10], and satellite repair [11], and are likely to be beneficial for mining as well.…”

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction: A Survey and Virtual Design Element Taxonomy

“…Entity Locations highlight the locations of entities within the robot's environment through rings, arrows, bounding boxes, etc. This VDE is especially useful when the location of an entity is occluded by walls or containers or outside of the user's field-of-view [19]. These can also be used to highlight task-and dialogue-relevant entities, either by allowing [49], E: Sensed Spatial Regions [69], F: Robot Inherent Spatial Regions [23], G: User-Defined Spatial Regions [78], H: Entity Labels, Entity Locations, and Task Status [8].…”