Surgical Tool Datasets for Machine Learning Research: A Survey

Abstract

This paper is a comprehensive survey of datasets for surgical tool detection and related surgical data science and machine learning techniques and algorithms. The survey offers a high level perspective of current research in this area, analyses the taxonomy of approaches adopted by researchers using surgical tool datasets, and addresses key areas of research, such as the datasets used, evaluation metrics applied and deep learning techniques utilised. Our presentation and taxonomy provides a framework that facilitates greater understanding of current work, and highlights the challenges and opportunities for further innovative and useful research.

Convolutional Neural Network‐Based Deep Learning Engine for Mastoidectomy Instrument Recognition and Movement Tracking

“…25 Data sets ideally should be representative of real-world surgery and include unbiased recordings and annotations, but to date many published data sets do not include variation in angles, magnification, clarity, illumination, and orientation. 26 Building data sets from recordings of mastoidectomies performed by residents in a live environment, as was done in this study, overcomes many of these biases but also introduces challenges with open-access publication of protected health information which can be institutionspecific. Additionally, the quality of data sets is affected by the quality and focus of annotations.…”

“…Strategies for annotations have varied significantly within and across surgical specialties, which can diminish the clinical utility of any identified objective characterization and make pooling of annotations or data sets challenging. 26 Though the current study utilized only categorization of 2 surgical instruments, future work should include annotations of anatomic landmarks and surgical phases to improve clinical relevance. A surgical phase is considered the highest temporal level component of an operation for segmentation purposes and can be divided into access, execution of objectives, and closure.…”

“…In general there is a paucity of large, heterogenous and high quality data sets across all surgical subspecialties, and only 1 has been published for otolaryngology 25 . Data sets ideally should be representative of real‐world surgery and include unbiased recordings and annotations, but to date many published data sets do not include variation in angles, magnification, clarity, illumination, and orientation 26 . Building data sets from recordings of mastoidectomies performed by residents in a live environment, as was done in this study, overcomes many of these biases but also introduces challenges with open‐access publication of protected health information which can be institution‐specific.…”

Effect of head-mounted displays on students’ acquisition of surgical suturing techniques compared to an e-learning and tutor-led course: a randomized controlled trial

“…In this regard, artificial intelligence (AI) algorithms for automated surgical performance assessment have already been used very successfully for endoscopic videos based on public datasets 54 , 56 . However, we are not aware of any public dataset relevant to the evaluation of open surgical sutures such as those presented in our study 57 . Therefore, as part of this study, we have published our dataset ( https://doi.org/10.5281/zenodo.7940583 ) for further development.…”

The recognition of surgical tools based on attention mechanism

“…A study [25] on datasets for medical tools shows that there are currently public datasets for medical tools, all of which from videos of surgical procedures used. In these videos, the surgical tools only have local targets, which is not conducive to solving the problem of tool identification in preoperative and postoperative tool management.…”