Initial Experiments with the Leap Motion as a User Interface in Robotic Endonasal Surgery

Travaglini, T. A.; Swaney, Philip J.; Weaver, Kyle D.; Webster, Robert J.

doi:10.1007/978-3-319-22368-1_17

Cited by 20 publications

(13 citation statements)

References 28 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, a simulation study of endonasal pituitary surgery that compared the LMC with the Phantom Omni showed that surgeons achieved a percentage of tumour mass resection and procedure duration very similar to that accomplished with the robot using the LMC (104).…”

Section: Discussionmentioning

confidence: 97%

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2018

Preprint

View full text Add to dashboard Cite

Background. The increasingly pervasive presence of technology in the operating room (OR) raises the need to study the interaction between the surgeon and the computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices that enable non-contact gesture-based human-computer interaction (HCI) are currently being explored as a solution in surgical environments. The aim of this systematic review was to provide an account of the state-of-the-art of COTS devices in the detection of manual gestures in surgery, and to identify their use as a simulation tool for teaching motor skills in minimally invasive surgery (MIS). Methods. A systematic literature review was conducted in PubMed, Embase, ScienceDirect and IEEE for articles published between January 2000 and 2016 on the use of COTS devices for gesture detection in surgical environments, and in simulation for surgical skills learning in MIS. Results. A total of 2709 studies were identified, 76 of which met the search selection criteria. The Microsoft KinectTM and the Leap Motion ControllerTM were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes; the possibility of using this technology to develop portable, low-cost simulators for skills learning in MIS was also examined. Given that the vast majority of articles found in this systematic review were proof-of-concept or prototype user and feasibility testing, we can conclude that this is a field that is still in the exploration phase in areas that require touchless manipulation in environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms. Conclusions: COTS devices applied to hand and instrument GBIs in the field of simulation for skills learning and training in MIS could open up a promising field to achieve the ubiquitous training and pre-surgical warm-up.

show abstract

Section: Discussionmentioning

confidence: 97%

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Most of the methods of learning human actions operate at the level of configuration-space trajectories [8] [9] [10]. Some researchers used kinesthetic teaching as the approach to provide demonstrations for a robot to learn from demonstrations [2] [11] [12].…”

Section: Related Workmentioning

confidence: 99%

Learning Actions from Human Demonstration Video for Robotic Manipulation

Yang

Zhang

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Learning actions from human demonstration is an emerging trend for designing intelligent robotic systems, which can be referred as video to command. The performance of such approach highly relies on the quality of video captioning. However, the general video captioning methods focus more on the understanding of the full frame, lacking of consideration on the specific object of interests in robotic manipulations. We propose a novel deep model to learn actions from human demonstration video for robotic manipulation. It consists of two deep networks, grasp detection network (GNet) and video captioning network (CNet). GNet performs two functions: providing grasp solutions and extracting the local features for the object of interests in robotic manipulation. CNet outputs the captioning results by fusing the features of both full frames and local objects. Experimental results on UR5 robotic arm show that our method could produce more accurate command from video demonstration than state-of-the-art work, thereby leading to more robust grasping performance.

show abstract

“…Brown et al [ 73 ] found a take-home laparoscopic instrument simulator for surgical education, while Cobb et al [ 74 ] present a survey on simulation in neurosurgery in connection with the use of the Kinect or LMC sensor. The LMC is used in conjunction with a haptic device (Phantom Omni) within a surgical simulation [ 75 ]. Especially, myographic gesture-control devices, like the Myo-band, will now also be important, for example, as an interaction solution for upper limb amputees [ 76 ] or as an interaction device in a physiotherapy treatment session [ 77 , 78 ].…”

Section: Applications and Contextsmentioning

confidence: 99%

Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller

Bachmann

Weichert

Rinkenauer

2018

Sensors

125

View full text Add to dashboard Cite

Modern hardware and software development has led to an evolution of user interfaces from command-line to natural user interfaces for virtual immersive environments. Gestures imitating real-world interaction tasks increasingly replace classical two-dimensional interfaces based on Windows/Icons/Menus/Pointers (WIMP) or touch metaphors. Thus, the purpose of this paper is to survey the state-of-the-art Human-Computer Interaction (HCI) techniques with a focus on the special field of three-dimensional interaction. This includes an overview of currently available interaction devices, their applications of usage and underlying methods for gesture design and recognition. Focus is on interfaces based on the Leap Motion Controller (LMC) and corresponding methods of gesture design and recognition. Further, a review of evaluation methods for the proposed natural user interfaces is given.

show abstract

Initial Experiments with the Leap Motion as a User Interface in Robotic Endonasal Surgery

Cited by 20 publications

References 28 publications

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Learning Actions from Human Demonstration Video for Robotic Manipulation

Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller

Contact Info

Product

Resources

About