Touch- and marker-free interaction with medical software

Kipshagen, Till; Graw, Mathis; Tronnier, V.; Bonsanto, Matteo M.; Hofmann, Urs

doi:10.1007/978-3-642-03906-5_21

Cited by 24 publications

(16 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Other groups have since reported similar results. 7 More recently, Wachs and colleagues [8][9][10] developed another gesture-based system using a simple webcam for data capture. Without gathering depth information, their group was able to devise a series of gestures to operate a PACS system and manipulate images intraoperatively.…”

Section: Discussionmentioning

confidence: 99%

Using a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field

Strickland

Tremaine²,

Brigley³

et al. 2013

Can J Surg

View full text Add to dashboard Cite

E1 RESEARCH • RECHERCHEUsing a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field Background: As surgical procedures become increasingly dependent on equipment and imaging, the need for sterile members of the surgical team to have unimpeded access to the nonsterile technology in their operating room (OR) is of growing import ance. To our knowledge, our team is the first to use an inexpensive infrared depthsensing camera (a component of the Microsoft Kinect) and software developed inhouse to give surgeons a touchless, gestural interface with which to navigate their picture archiving and communication systems intraoperatively. Methods:The system was designed and developed with feedback from surgeons and OR personnel and with consideration of the principles of aseptic technique and gestural controls in mind. Simulation was used for basic validation before trialing in a pilot series of 6 hepatobiliary-pancreatic surgeries. Results:The interface was used extensively in 2 laparoscopic and 4 open procedures. Surgeons primarily used the system for anatomic correlation, real-time comparison of intraoperative ultrasound with preoperative computed tomography and magnetic resonance imaging scans and for teaching residents and fellows.

show abstract

Section: Discussionmentioning

confidence: 99%

Using a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field

Strickland

Tremaine²,

Brigley³

et al. 2013

Can J Surg

View full text Add to dashboard Cite

show abstract

“…Gesture recognition systems with web and video cameras were later described [26,27], using the Time-Of-Flight principle [28] and achieving interaction with the OsiriX viewer [19,29]. These systems, though, were very expensive, inaccurate and required calibration and complex set up, making them impractical for use in the OR [30].…”

Section: Introductionmentioning

confidence: 99%

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2018

Preprint

View full text Add to dashboard Cite

Background. The increasingly pervasive presence of technology in the operating room (OR) raises the need to study the interaction between the surgeon and the computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices that enable non-contact gesture-based human-computer interaction (HCI) are currently being explored as a solution in surgical environments. The aim of this systematic review was to provide an account of the state-of-the-art of COTS devices in the detection of manual gestures in surgery, and to identify their use as a simulation tool for teaching motor skills in minimally invasive surgery (MIS). Methods. A systematic literature review was conducted in PubMed, Embase, ScienceDirect and IEEE for articles published between January 2000 and 2016 on the use of COTS devices for gesture detection in surgical environments, and in simulation for surgical skills learning in MIS. Results. A total of 2709 studies were identified, 76 of which met the search selection criteria. The Microsoft KinectTM and the Leap Motion ControllerTM were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes; the possibility of using this technology to develop portable, low-cost simulators for skills learning in MIS was also examined. Given that the vast majority of articles found in this systematic review were proof-of-concept or prototype user and feasibility testing, we can conclude that this is a field that is still in the exploration phase in areas that require touchless manipulation in environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms. Conclusions: COTS devices applied to hand and instrument GBIs in the field of simulation for skills learning and training in MIS could open up a promising field to achieve the ubiquitous training and pre-surgical warm-up.

show abstract

“…Although algorithms that capture full skeletal motion at near real-time frame rates are available, the special controlled recording conditions required make them unsuitable for use in operating rooms. In [8] the WagO system, a hand gesture control plug-in for the open source DICOM viewer OsiriX [11], was presented. Hand gestures and 3D hand positions are recognized by using two webcams in stereo configuration.…”

Section: Introductionmentioning

confidence: 99%