Remote Presence: Development and Usability Evaluation of a Head-Mounted Display for Camera Control on the da Vinci Surgical System

Abstract

This paper describes the development of a new method to control the camera arm of a surgical robot and create a better sense of remote presence for the surgeon. The current surgical systems are entirely controlled by the surgeon, using hand controllers and foot pedals to manipulate either the instrument or the camera arms. The surgeon must pause the operation to move the camera arm to obtain a desired view and then resume the operation. The camera and tools cannot be moved simultaneously, leading to interrupted and unnatural movements. These interruptions can lead to medical errors and extended operation times. In our system, the surgeon controls the camera arm by his natural head movements while being immersed in a 3D-stereo view of the scene with a head-mounted display (HMD). The novel approach enables the camera arm to be maneuvered based on sensors of the HMD. We implemented this method on a da Vinci Standard Surgical System using the HTC Vive headset along with the Unity engine and the Robot Operating System framework. This paper includes the result of a subjective six-participant usability study that compares the workload of the traditional clutched camera control method against the HMD-based control. Initial results indicate that the system is usable, stable, and has a lower physical and mental workload when using the HMD control method.

Development & Evaluation of Virtual Reality (VR) based Novel Vision System for the da Vinci Surgical Robot

“…According to most results of the peg transfer tasks, the VR HMD displayed improved results compared to the stereo viewer, as summarized in Tables 3 and 4. In particular, there was reduction in the average execution times and an increase in the total number of executions that resembles the results of Jo et al and Dardona et al [28,29]. In the VR HMD, the average execution time decreased by up to 21.18%, and the total number of executions increased by up to 21.02%, which makes the difference larger as the stage progresses (Tables 3 and 4).…”

“…Further, the collision avoidance algorithm was evaluated to demonstrate that the algorithm can prevent collisions between the surgical tools and the endoscope when the intention of the surgeon is blended with a compensation signal in a proper ratio. Dardona et al performed a comparison of the control interfaces for the ECS between the VR HMD and a conventional clutched method [29]. The participants indicated that the VR HMD is easier to learn and it is comfortable since it does not cause dizziness or physical inconvenience.…”

Scheduling and Path-Planning for Operator Oversight of Multiple Robots

“…A study case of a foraging task is presented in [27] where queuing techniques were used to schedule the operator's attention. In [28], Dardona et al have shown that the output side sensor configuration of a teleoperated system is altered to reduce the workload of a remote surgeon.…”

Development and evaluation of an autonomous camera control algorithm on the da Vinci Surgical System

“…9 A method of controlling the da Vinci system's camera arm using a headmounted display was developed with our test platform (the one described in this paper), and a usability study was performed. 10 There are several automated camera control approaches given in the literature, including reactive, proactive, or combined control strategies. Reactive approaches track data such as tool positions or eye gaze and move the camera in direct response to changes in these inputs.…”

“…Ji et al used a segmentation algorithm to create optimum camera viewpoints . A method of controlling the da Vinci system's camera arm using a head‐mounted display was developed with our test platform (the one described in this paper), and a usability study was performed …”