No abstract

Background Artificial Intelligence (AI) holds tremendous potential to reduce surgical risks and improve surgical assessment. Machine Learning, a subfield of AI, relies on video and image data, where annotations provide veracity about the desired target features. Yet, methodological annotation explorations are limited to date. Here, we provide an exploratory analysis of the requirements and methods of instrument annotation in a multi-institutional team from two specialized AI centers and compile a structured manual for future AI projects focusing on instrument detection. Methods We developed a bottom-up approach for team annotation of robotic instruments in robotassisted partial nephrectomy (RAPN), after which it was validated in robot-assisted minimally invasive esophagectomy (RAMIE). Furthermore, instrument annotation methods were evaluated for their use in Machine Learning algorithms. Overall, we evaluated the efficiency and transferability of the proposed team approach and quantified performance metrics (e.g. time per frame required for each annotation modality) between RAPN and RAMIE. ResultsThe proposed annotation methodology was transferrable between both RAPN and RAMIE. The bottom-up approach of annotation management and training resulted in accurate annotations and demonstrated efficiency in annotating large datasets and diverse annotator groups. The average annotation time for RAPN for pixel annotation ranged from 4.49 to 12.6 minutes per image; for vector annotation this was decreased to 2.92 minutes. Similar ranges of pixel annotation times were denoted for RAMIE. Lastly, we elaborate on common pitfalls encountered throughout the annotation process. Conclusions We propose a successful bottom-up approach for annotator team composition, applicable to any annotation project. Our results set the foundation to start AI projects for instrument detection, segmentation and pose estimation. Due to the immense annotation burden resulting from spatial instrumental annotation, further analysis into sampling frequency and annotation detail needs to be conducted.