The Effects of Visual and Control Latency on Piloting a Quadcopter Using a Head-Mounted Display

Abstract

Recent research has proposed teleoperation of robotic and aerial vehicles using head motion tracked by a headmounted display (HMD). First-person views of the vehicles are usually captured by onboard cameras and presented to users through the display panels of HMDs. This provides users with a direct, immersive and intuitive interface for viewing and control. However, a typically overlooked factor in such designs is the latency introduced by the vehicle dynamics. As head motion is coupled with visual updates in such applications, visual and control latency always exists between the issue of control commands by head movements and the visual feedback received at the completion of the attitude adjustment. This causes a discrepancy between the intended motion, the vestibular cue and the visual cue and may potentially result in simulator sickness. No research has been conducted on how various levels of visual and control latency introduced by dynamics in robots or aerial vehiclesaffect users' performance and the degree of simulator sickness elicited. Thus, it is uncertain how much performance is degraded by latency and whether such designs are comfortable from the perspective of users. To address these issues, we studied a prototyped scenario of a head motion controlled quadcopter using an HMD. We present a virtual reality (VR) paradigm to systematically assess the effects of visual and control latency in simulated drone control scenarios.

Comparing head gesture, hand gesture and gamepad interfaces for answering Yes/No questions in virtual environments

“…These include treadmills [28], [29], foot platforms [30], pedalling devices [31] and spheres [32], etc. Finally, several studies have proposed the use of head motions for teleoperation of robots or for virtual locomotion [33], [34], [35], [36].…”

“…We adapted this experiment from [35] to evaluate the usability of interfaces for waypoints navigation. In this experiment, we used the same terrain of Experiment 1 but placed wood gates as the waypoints in the virtual environment (shown in Figure 11).…”

“…Based on the metrics given in [35], we proposed the following metrics to evaluate the interfaces for our study:…”

A Systematic Review of Virtual Reality Interfaces for Controlling and Interacting with Robots

“…Similarly, ref. [27] analyzes the effects of visual and control latency in drones when using VR. Unsurprisingly, they find that an increase in latency resulted in worse flight performance and a higher level of motion sickness.…”