Mental Transformations in Human-Robot Interaction

DeJong, Brian Peter; Colgate, J. Edward; Peshkin, Michael A.

doi:10.1007/978-94-007-0582-1_3

Cited by 21 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These solutions are fundamentally insufficient, however. Even if surgeons had convenient and ergonomic VR interactivity, they would find themselves limited to small visualization changes while operating because large view rotations would sever their intuitive connection to their tools [43]. Consequently, surgeons would suffer from misalignment between their hands and viewed tool motions, offsetting the benefits of intraoperative volumetric imaging.…”

Section: Discussionmentioning

confidence: 99%

Real-time visualization and interaction with static and live optical coherence tomography volumes in immersive virtual reality

Draelos

Keller

Viehland

et al. 2018

Biomed. Opt. Express

View full text Add to dashboard Cite

Virtual reality (VR) head-mounted displays are an attractive technology for viewing intrasurgical optical coherence tomography (OCT) volumes because they liberate surgeons from microscope oculars. We demonstrate real-time, interactive viewing of OCT volumes in a commercial HTC Vive immersive VR system using previously reported ray casting techniques. Furthermore, we show interactive manipulation and sectioning of volumes using handheld controllers and guidance of mock surgical procedures in porcine eyes exclusively within VR. To the best of our knowledge, we report the first immersive VR-OCT viewer with stereo ray casting volumetric renders, arbitrary sectioning planes, and live acquisition support. We believe VR-OCT volume displays will advance ophthalmic surgery towards VR-integrated surgery.

show abstract

Section: Discussionmentioning

confidence: 99%

Real-time visualization and interaction with static and live optical coherence tomography volumes in immersive virtual reality

Draelos

Keller

Viehland

et al. 2018

Biomed. Opt. Express

View full text Add to dashboard Cite

show abstract

“…More correction methods are in improving the view display. For example, Dejone et al [21], [22] proposed to reduce the teleoperation mental workload by optimizing component arrangement in an interface. 2.…”

Section: A Related Workmentioning

confidence: 99%

Camera Frame Misalignment in a Teleoperated Eye-in-Hand Robot: Effects and a Simple Correction Method

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Misalignment between the camera frame and the operator frame is commonly seen in a teleoperated system and usually degrades the operation performance. The effects of such misalignment have not been fully investigated for eye-in-hand systems -systems that have the camera (eye) mounted to the end-effector (hand) to gain compactness in confined spaces such as in endoscopic surgery. This paper provides a systematic study on the effects of the camera frame misalignment in a teleoperated eye-in-hand robot and proposes a simple correction method in the view display. A simulation is designed to compare the effects of the misalignment under different conditions. Users are asked to move a rigid body from its initial position to the specified target position via teleoperation, with different levels of misalignment simulated. It is found that misalignment between the input motion and the output view is much more difficult to compensate by the operators when it is in the orthogonal direction (∼40s) compared with the opposite direction (∼20s). An experiment on a real concentric tube robot with an eye-in-hand configuration is also conducted. Users are asked to telemanipulate the robot to complete a pick-and-place task. Results show that with the correction enabled, there is a significant improvement in the operation performance in terms of completion time (mean 40.6%, median 38.6%), trajectory length (mean 34.3%, median 28.1%), difficulty (50.5%), unsteadiness (49.4%), and mental stress (60.9%).

show abstract

“…Only few human-robot studies took gaze location into consideration for improving human task performance (DeJong et al, 2011), which involved improving human performance in spatial transformations using robotic arms.…”

Section: Gaze Control: Computer Robot and Swarm Controlmentioning

confidence: 99%

Human Strategies for Multitasking, Search, and Control Improved via Real-Time Memory Aid for Gaze Location

Taylor

Bilgrien

et al. 2015

Front. ICT

View full text Add to dashboard Cite

Objective: We aimed to elucidate how our domain-general cuing algorithm improved multitasking performance and changed behavioral strategies in human operators.Background: Though many gaze-control systems have been designed, previous realtime gaze-aware assistance systems were not both successful and domain-general. It is largely unknown what constitutes optimal search efficiency using the eyes, or ideal control using the mouse. It is unclear what the best coordinating strategies are between these two modalities. Our previously developed closed-loop multitasking aid drastically improved multitasking performance, though the behavioral mechanisms through which it acted were unknown. Methods:We performed in-depth analyses and generated novel eye tracking and mouse movement measures, to explore the complex effects of our helpful system on gaze and motor behavior.Results: Our overlay cuing algorithm improved control efficiency and reduced wellknown biases in search patterns. This system also reduced micromanaging behavior, with humans rationally relying more on imperfect automation in experimental assistance cue conditions. We showed that mouse and gaze were more independently specialized in the helpful cuing condition than in control conditions. Specifically, with our aid, the gaze performed more global movement, and the mouse performed more local clustered movement. Further, the gaze shifted toward search over processing with the helpful cuing system. We also illustrated a relationship between the mouse and the gaze, such that in these studies, "the hand was quicker than the eye." Conclusion:Overall, results suggested that our cuing system improved performance and reduced short-term working memory load on humans by delegating it to the computer in real time. Further, it reduced the number of required repeated decisions by an estimate of about one per second. It also enabled the gaze to specialize for improved visual search behavior, and the mouse to specialize for improved control.

show abstract

Mental Transformations in Human-Robot Interaction

Cited by 21 publications

References 24 publications

Real-time visualization and interaction with static and live optical coherence tomography volumes in immersive virtual reality

Real-time visualization and interaction with static and live optical coherence tomography volumes in immersive virtual reality

Camera Frame Misalignment in a Teleoperated Eye-in-Hand Robot: Effects and a Simple Correction Method

Human Strategies for Multitasking, Search, and Control Improved via Real-Time Memory Aid for Gaze Location

Contact Info

Product

Resources

About