Inductive Learning of the Surgical Workflow Model through Video Annotations

Abstract

Surgical workflow modeling is becoming increasingly useful to train surgical residents for complex surgical procedures. Rule-based surgical workflows have shown to be useful to create context-aware systems. However, manually constructing production rules is a time-intensive and laborious task. With the expansion of new technologies, large video archive can be created and annotated exploiting and storing the expert's knowledge. This paper presents a prototypical study of automatic generation of production rules, in the Horn-clause, using the First Order Inductive Learner (FOIL) algorithm applied to annotated surgical videos of Thoracentesis procedure and of its feasibility to use in context-aware system framework. The algorithm was able to learn 18 rules for surgical workflow model with 0.88 precision, and 0.94 F1 score on the standard video annotation data, representing entities of the surgical workflow, which was used to retrieve contextual information on Thoracentesis workflow for its application to surgical training.

“Deep-Onto” network for surgical workflow and context recognition

“…Moreover, "Anatomy" which was currently grounded explicitly in the production rules if recognized could be used as a pre-condition as a realtime context recognition. As it is hypothesized that a context for the recognition of surgical workflow would be different at each step of the surgery, automatic generation of production rules, e.g., with inductive learning [34], could provide extended capability for adaptive learning on realworld instances.…”

Toward a standard ontology of surgical process models

“…The system was deployed for context-aware surgical training, where the ontology-based system gave the similar results as mentor-based surgical training. Furthermore, production rules were created automatically using the first-order inductive learning [51], which were used in task execution. These contributions can be reused to address the need of integrating entities representing image processing and robotic components within a common ontology of SPM, which could be useful in robotic surgery.…”