Image-based tracking of medical instruments is an integral part of surgical data science applications. Previous research has addressed the tasks of detecting, segmenting and tracking medical instruments based on laparoscopic video data. However, the proposed methods still tend to fail when applied to challenging images and do not generalize well to data they have not been trained on. This paper introduces the Heidelberg Colorectal (HeiCo) data set - the first publicly available data set enabling comprehensive benchmarking of medical instrument detection and segmentation algorithms with a specific emphasis on method robustness and generalization capabilities. Our data set comprises 30 laparoscopic videos and corresponding sensor data from medical devices in the operating room for three different types of laparoscopic surgery. Annotations include surgical phase labels for all video frames as well as information on instrument presence and corresponding instance-wise segmentation masks for surgical instruments (if any) in more than 10,000 individual frames. The data has successfully been used to organize international competitions within the Endoscopic Vision Challenges 2017 and 2019.

Semantic segmentation of organs and tissue types is an important sub-problem in image based scene understanding for laparoscopic surgery and is a prerequisite for context-aware assistance and cognitive robotics. Deep Learning (DL) approaches are prominently applied to segmentation and tracking of laparoscopic instruments. This work compares different combinations of neural networks, loss functions, and training strategies in their application to semantic segmentation of different organs and tissue types in human laparoscopic images in order to investigate their applicability as components in cognitive systems. TernausNet-11 trained on Soft-Jaccard loss with a pretrained, trainable encoder performs best in regard to segmentation quality (78.31% mean Intersection over Union [IoU]) and inference time (28.07 ms) on a single GTX 1070 GPU.

No abstract

No abstract

No abstract

No abstract

Purpose Endovascular interventions require intense practice to develop sufficient dexterity in catheter handling within the human body. Therefore, we present a modular training platform, featuring 3D-printed vessel phantoms with patient-specific anatomy and integrated piezoresistive impact force sensing of instrument interaction at clinically relevant locations for feedback-based skill training to detect and reduce damage to the delicate vascular wall. Methods The platform was fabricated and then evaluated in a user study by medical ($$n=10$$ n = 10 ) and non-medical ($$n=10$$ n = 10 ) users. The users had to navigate a set of guidewire and catheter through a parkour of 3 modules including an aneurismatic abdominal aorta, while impact force and completion time were recorded. Eventually, a questionnaire was conducted. Results The platform allowed to perform more than 100 runs in which it proved capable to distinguish between users of different experience levels. Medical experts in the fields of vascular and visceral surgery had a strong performance assessment on the platform. It could be shown, that medical students could improve runtime and impact over 5 runs. The platform was well received and rated as promising for medical education despite the experience of higher friction compared to real human vessels. Conclusion We investigated an authentic patient-specific training platform with integrated sensor-based feedback functionality for individual skill training in endovascular surgery. The presented method for phantom manufacturing is easily applicable to arbitrary patient-individual imaging data. Further work shall address the implementation of smaller vessel branches, as well as real-time feedback and camera imaging for further improved training experience.

No abstract