Latency impact on Quality of Experience in a virtual reality simulator for remote control of machines

Abstract

In this article, we have investigated a VR simulator of a forestry crane used for loading logs onto a truck. We have mainly studied the Quality of Experience (QoE) aspects that may be relevant for task completion, and whether there are any discomfort related symptoms experienced during the task execution. QoE experiments were designed to capture the general subjective experience of using the simulator, and to study task performance. The focus was to study the effects of latency on the subjective experience, with regards to delays in the crane control interface. Subjective studies were performed with controlled delays added to the display update and hand controller (joystick) signals. The added delays ranged from 0 to 30 ms for the display update, and from 0 to 800 ms for the hand controller. We found a strong effect on latency in the display update and a significant negative effect for 800 ms added delay on latency in the hand controller (in total approx. 880 ms latency including the system delay). The Simulator Sickness Questionnaire (SSQ) gave significantly higher scores after the experiment compared to before the experiment, but a majority of the participants reported experiencing only minor symptoms. Some test subjects ceased the test before finishing due to their symptoms, particularly due to the added latency in the display update.

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

“…In other non-entertainment contexts, augmented remote operation appears to benefit the operators in terms of their Quality of Experience (QoE) and task accomplishment, as demonstrated in [6,8,9,10,11]. The augmentation covered by our system is a relatively simple view manipulation, meant to explore the technical feasibility more than the specific operator experience.…”

“…Due to the applied nature of the mining industry, there usually is a resistance to high-level concepts, and a desire for high-feasibility solutions with a high technical readiness level. The applied state of the art for remote operation in mines is direct image (video) transmission, similar to the systems described in [4,5,6]. Fig.…”

“…However, going beyond the direct presentation of camera views, there is potential in using AR, view synthesis, and range sensing from Time-of-Flight (ToF) sensors such as Light Detection And Ranging (lidar) to further enhance the operator awareness of the on-site environment [7,8]. The benefits of augmented and indirect views have been investigated in other contexts such as underwater robot operation [9], forestry [6], hazard exploration [10], and satellite repair [11], and are likely to be beneficial for mining as well.…”

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practises for QoE Assessment

“…Additionally, there are some few works which are focused more on interaction rather than on communication (Brunnström et al, 2020;Pérez et al, 2021). Still, it is still interesting to understand their related evaluation methodologies for hybrid immersive systems where both communication and interaction are important.…”

“…Nevertheless, Social XR/VR systems offer the possibility to test the interact with the environment, such as in Li et al (2021), where the users could press buttons, or with virtual objects, such as in Pan and Steed (2017) and Prins et al (2018), where the users played together puzzles and pong games, respectively. In this sense, it is also worth mentioning the insights that can provide user studies focused on analyzing the interaction of the users with the systems without accounting for communication, such as the one carried out by Brunnström et al (2020) with a crane-control system and by Pérez et al (2021) considering a scape room scenario.…”