Exploring the potential for touchless interaction in image-guided interventional radiology

Johnson, Rose; O’Hara, Kenton; Sellen, Abigail; Cousins, Claire; Criminisi, Antonio

doi:10.1145/1978942.1979436

Cited by 109 publications

(57 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ethnographic studies [41][42][43][44]76] deserve a separate mention, since they transcend proofs-of-concept and user and prototype testing, and approach gesture-based touchless interaction from a holistic view that includes the social practices of surgery, and how the medical images and manipulation devices are embedded and made meaningful within the collaborative practices of the surgery [10].…”

Section: Discussionmentioning

confidence: 99%

“…Most of the articles (47) identified in this systematic review were proof-of-concept or prototype user and feasibility testing ( Table 1). The five ethnographic studies retained had the aim of identifying interactions between personnel and the gesture-based COTS systems in interventional radiology departments or in the OR [16,[41][42][43][44]. Two studies were aimed at comparing the performance of the MK with respect to a mouse [45,46]; one compared the performance of the LMC to that of a mouse [47]; and two studies compared different COTS devices [48,49].…”

Section: User Testingmentioning

confidence: 99%

See 1 more Smart Citation

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2018

Preprint

View full text Add to dashboard Cite

Background. The increasingly pervasive presence of technology in the operating room (OR) raises the need to study the interaction between the surgeon and the computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices that enable non-contact gesture-based human-computer interaction (HCI) are currently being explored as a solution in surgical environments. The aim of this systematic review was to provide an account of the state-of-the-art of COTS devices in the detection of manual gestures in surgery, and to identify their use as a simulation tool for teaching motor skills in minimally invasive surgery (MIS). Methods. A systematic literature review was conducted in PubMed, Embase, ScienceDirect and IEEE for articles published between January 2000 and 2016 on the use of COTS devices for gesture detection in surgical environments, and in simulation for surgical skills learning in MIS. Results. A total of 2709 studies were identified, 76 of which met the search selection criteria. The Microsoft KinectTM and the Leap Motion ControllerTM were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes; the possibility of using this technology to develop portable, low-cost simulators for skills learning in MIS was also examined. Given that the vast majority of articles found in this systematic review were proof-of-concept or prototype user and feasibility testing, we can conclude that this is a field that is still in the exploration phase in areas that require touchless manipulation in environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms. Conclusions: COTS devices applied to hand and instrument GBIs in the field of simulation for skills learning and training in MIS could open up a promising field to achieve the ubiquitous training and pre-surgical warm-up.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: User Testingmentioning

confidence: 99%

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Another group has recently reported using similar hardware to extract data for gestural interpretation for possible use in surgical settings, 21 but to the best of our knowledge, the work we describe here is the first time an IR depthsensing camera has been used intraoperatively as part of a human-machine interface. Furthermore, it is the first time a vision-based system has been used so extensively for image navigation outside of stereotactic surgery.…”

Section: Discussionmentioning

confidence: 99%

Using a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field

Strickland

Tremaine²,

Brigley³

et al. 2013

Can J Surg

View full text Add to dashboard Cite

E1 RESEARCH • RECHERCHEUsing a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field Background: As surgical procedures become increasingly dependent on equipment and imaging, the need for sterile members of the surgical team to have unimpeded access to the nonsterile technology in their operating room (OR) is of growing import ance. To our knowledge, our team is the first to use an inexpensive infrared depthsensing camera (a component of the Microsoft Kinect) and software developed inhouse to give surgeons a touchless, gestural interface with which to navigate their picture archiving and communication systems intraoperatively. Methods:The system was designed and developed with feedback from surgeons and OR personnel and with consideration of the principles of aseptic technique and gestural controls in mind. Simulation was used for basic validation before trialing in a pilot series of 6 hepatobiliary-pancreatic surgeries. Results:The interface was used extensively in 2 laparoscopic and 4 open procedures. Surgeons primarily used the system for anatomic correlation, real-time comparison of intraoperative ultrasound with preoperative computed tomography and magnetic resonance imaging scans and for teaching residents and fellows.

show abstract

“…Trackers are generally used in surgical settings, but cost, line-of-sight requirement, and calibration (for optical tracking) and possible interference (for electromagnetic tracking) can cause problems. Furthermore, all these tactile interaction methods cause sterilization concerns when used in the operating room environment [20]. Gesture-based interaction eliminates the sterilization requirement because the camera can be placed in the non-sterile working area and no additional equipment is necessary.…”

Section: Interaction Methodsmentioning

confidence: 99%

Interactive Visualization and Analysis of Multimodal Datasets for Surgical Applications

et al. 2012

View full text Add to dashboard Cite

Surgeons use information from multiple sources when making surgical decisions. These include volumetric datasets (such as CT, PET, MRI, and their variants), 2D datasets (such as endoscopic videos), and vector-valued datasets (such as computer simulations). Presenting all the information to the user in an effective manner is a challenging problem. In this paper, we present a visualization approach that displays the information from various sources in a single coherent view. The system allows the user to explore and manipulate volumetric datasets, display analysis of dataset values in local regions, combine 2D and 3D imaging modalities and display results of vector-based computer simulations. Several interaction methods are discussed: in addition to traditional interfaces including mouse and trackers, gesture-based natural interaction methods are shown to control these visualizations with real-time performance. An example of a medical application (medialization laryngoplasty) is presented to demonstrate how the combination of different modalities can be used in a surgical setting with our approach.

show abstract

Exploring the potential for touchless interaction in image-guided interventional radiology

Cited by 109 publications

References 16 publications

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Use of commercial off-the-shelf devices for detection of manual gestures in surgery. A Systematic Literature Review (Preprint)

Using a depth-sensing infrared camera system to access and manipulate medical imaging from within the sterile operating field

Interactive Visualization and Analysis of Multimodal Datasets for Surgical Applications

Contact Info

Product

Resources

About