Gesture recognition system for surgical robot's manipulation

Vargas, Hermes F.; Albán, Óscar Andrés Vivas

doi:10.1109/stsiva.2014.7010172

Cited by 16 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HoloLens [11] In this paper, Leap Motion, as shown in Fig.2, is used to capture the human operator's hand motion and then generate position information for controlling the slave robot. Developed by Magic Leap Company, Leap Motion is able to trace the user's hands' trail by using infrared LEDs and depth cameras [12]. [13] Teleoperation belongs to Human-Robot interaction(HRI) [14], which has become a important topic.…”

Section: Introductionmentioning

confidence: 99%

Robot Teleoperation System Based on Mixed Reality

Liang

Liu

et al. 2019

2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM)

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Robot Teleoperation System Based on Mixed Reality

Liang

Liu

et al. 2019

2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM)

View full text Add to dashboard Cite

“…Studies on the application of gesture-based COTS devices in robot-assisted surgery failed to demonstrate usefulness, owing to either the high cost of the robotic arm when using commercial cameras in surgical instrumentation [115] or, in the case of the LMC, the need for a more robust Application Programming Interface [116,117] and the lack of sufficient accuracy and robustness for manipulating a medical robot [113]. However, an ethnographic study found that MK was useful for workflow monitoring and for avoiding collisions between medical robots and operating room staff [114].…”

Section: Discussionmentioning

confidence: 99%

“…In the field of simulation for robotic surgery learning, the first studies published [113,115-117] found that the interfaces did not allow robots to be manipulated by gestures. However, the most recent publications [118,119] have suggested that the LMC could be a low-cost solution for creating control interfaces for surgical robots for the purposes of performing operations or training by means of simulation.…”

Section: Discussionmentioning

confidence: 99%

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2019

J Med Internet Res

View full text Add to dashboard Cite

Background The increasingly pervasive presence of technology in the operating room raises the need to study the interaction between the surgeon and computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices enabling touchless gesture–based human-computer interaction is currently being explored as a solution in surgical environments. Objective The aim of this systematic literature review was to provide an account of the state of the art of COTS devices in the detection of manual gestures in surgery and to identify their use as a simulation tool for motor skills teaching in minimally invasive surgery (MIS). Methods For this systematic literature review, a search was conducted in PubMed, Excerpta Medica dataBASE, ScienceDirect, Espacenet, OpenGrey, and the Institute of Electrical and Electronics Engineers databases. Articles published between January 2000 and December 2017 on the use of COTS devices for gesture detection in surgical environments and in simulation for surgical skills learning in MIS were evaluated and selected. Results A total of 3180 studies were identified, 86 of which met the search selection criteria. Microsoft Kinect (Microsoft Corp) and the Leap Motion Controller (Leap Motion Inc) were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes. The possibility of using this technology to develop portable low-cost simulators for skills learning in MIS was also examined. As most of the articles identified in this systematic review were proof-of-concept or prototype user testing and feasibility testing studies, we concluded that the field was still in the exploratory phase in areas requiring touchless manipulation within environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms. Conclusions COTS devices applied to hand and instrument gesture–based interfaces in the field of simulation for skills learning and training in MIS could open up a promising field to achieve ubiquitous training and presurgical warm up.

show abstract

“…Toward creating less expensive master interfaces that are not subject to mechanical wear, there has been increasing recent interest in hand tracking as an alternative approach [15, 16, 30, 26, 7]. This trend has been spurred by the recent introduction of low-cost, consumer-based tracking devices such as the Kinect (Microsoft, Inc., USA) and the Leap Motion (Leap Motion, Inc., USA).…”

Section: Introductionmentioning

confidence: 99%

Initial Experiments with the Leap Motion as a User Interface in Robotic Endonasal Surgery

Travaglini

Swaney

Weaver

et al. 2015

Robotics and Mechatronics

View full text Add to dashboard Cite

The Leap Motion controller is a low-cost, optically-based hand tracking system that has recently been introduced on the consumer market. Prior studies have investigated its precision and accuracy, toward evaluating its usefulness as a surgical robot master interface. Yet due to the diversity of potential slave robots and surgical procedures, as well as the dynamic nature of surgery, it is challenging to make general conclusions from published accuracy and precision data. Thus, our goal in this paper is to explore the use of the Leap in the specific scenario of endonasal pituitary surgery. We use it to control a concentric tube continuum robot in a phantom study, and compare user performance using the Leap to previously published results using the Phantom Omni. We find that the users were able to achieve nearly identical average resection percentage and overall surgical duration with the Leap.

show abstract

Gesture recognition system for surgical robot's manipulation

Cited by 16 publications

References 9 publications

Robot Teleoperation System Based on Mixed Reality

Robot Teleoperation System Based on Mixed Reality

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Initial Experiments with the Leap Motion as a User Interface in Robotic Endonasal Surgery

Contact Info

Product

Resources

About