Development of an intelligent surgical training system for Thoracentesis

Abstract

Surgical training improves patient care, helps to reduce surgical risks, increases surgeon's confidence, and thus enhances overall patient safety. Current surgical training systems are more focused on developing technical skills, e.g. dexterity, of the surgeons while lacking the aspects of context-awareness and intra-operative real-time guidance. Context-aware intelligent training systems interpret the current surgical situation and help surgeons to train on surgical tasks. As a prototypical scenario, we chose Thoracentesis procedure in this work. We designed the context-aware software framework using the surgical process model encompassing ontology and production rules, based on the procedure descriptions obtained through textbooks and interviews, and ontology-based and marker-based object recognition, where the system tracked and recognised surgical instruments and materials in surgeon's hands and recognised surgical instruments on the surgical stand. The ontology was validated using annotated surgical videos, where the system identified "Anaesthesia" and "Aspiration" phase with 100% relative frequency and "Penetration" phase with 65% relative frequency. The system tracked surgical swab and 50mL syringe with approximately 88.23% and 100% accuracy in surgeon's hands and recognised surgical instruments with approximately 90% accuracy on the surgical stand. Surgical workflow training with the proposed system showed equivalent results as the traditional mentor-based training regime, thus this work is a step forward a new tool for context awareness and decision-making during surgical training.

On strategic choices faced by large pharmaceutical laboratories and their effect on innovation risk under fuzzy conditions

“…The application of fuzzy methodologies to the healthcare industries has been proposed recently by several authors. For example, Albino et al (2018) and Nakawala et al (2018) employ this approach to improve process and enhance training at hospitals. A different line of research is that by Carlsson et al (2007) andLo Nigro et al (2016), which suggest the usefulness of fuzzy techniques to optimize the R&D portfolio.…”

“…Prior fuzzy works have shown several applications of fuzzy methodologies in the medical and pharmaceutical industries. Some researchers use this approach to improve medical procedures and training at hospitals, such as Mendez et al (2018) and Nakawala et al (2018). From a different prism, Gascón et al (2007) classify countries depending on their likelihood to consume and produce generics through fuzzy techniques.…”

Approaches for Action Sequence Representation in Robotics: A Review

“…In this work, the robot uses effect axioms to predict the outcome of actions as the resulting world state. Similarly, the ontology and production rules have been used in [25] for the execution planning in surgery based on the perceived instruments. However, the task execution has not been done by the robots.…”

“Deep-Onto” network for surgical workflow and context recognition

“…An ontology provides an explicit specification of concepts within a domain of interest, which could be used to represent "Surgical Process Model" (SPM). In the past, ontologies were used for "phase" recognition in laparoscopic surgeries [9] and context-aware training in percutaneous surgeries [10]. However, perceptual object, e.g., surgical tools in videos, is difficult to be recognized with knowledge-based techniques.…”

“…The "Deep-Onto" network is an ensemble of two components: (1) a "CRNN" and a "Sequence" model to recognize the surgical step and a subsequent step from RAPN videos and (2) a "Knowledge" model, which contains an ontology-based SPM on RAPN and logical rules to recognize other surgical context, e.g., instruments. The aim is to automatically understand RAPN workflow, which could be used in a context-aware system framework [10] and eventually assist novice surgeons during surgical training by presenting the contextual information, e.g., the next step during the intervention. As far as our knowledge allows, this is a first implementation of a combined use of deep learning, and knowledge representation and reasoning techniques for the automatic surgical workflow analysis on robot-assisted urological surgery.…”