Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

Abstract

The recent advancements in the surging field of Deep Learning (DL) have revolutionized every sphere of life, and the healthcare domain is no exception. The enormous success of DL models, particularly with image data, has led to the development of image-guided Robot Assisted Surgery (RAS) systems. By and large, the number of studies concerning image-driven computer assisted surgical systems using DL has increased exponentially. Additionally, the contemporary availability of surgical datasets has also boosted the DL applications in RAS. Inspired by the latest trends and contributions in surgery, this literature survey presents a summarized analysis of recent innovations of DL in image-guided RAS systems. After a thorough review, a sum of 184 articles are selected and grouped into four categories, based on the literature and the relevancy of the task in the articles, comprising 1) Surgical Tools, 2) Surgical Processes, 3) Surgical Surveillance, and 4) Surgical Performance. The survey also discusses publicly available surgical datasets and highlights the basics of the DL models. Furthermore, the legal, ethical, and technological challenges together with the intuitive predictions and recommendations related to the autonomous RAS systems are also presented. The study reveals that Convolutional Neural Network (CNN) is most widely adopted architecture, whereas, JIGSAWS is most employed dataset in RAS. The study suggests fusing kinematic data along with image data, which produces better accuracy and precision, particularly in gesture and trajectory segmentation tasks. Additionally, CNN and Long Short Term Memory (LSTM) networks have shown remarkable performance, however, authors recommend employing these gigantic architectures only when simpler models have failed to produce satisfactory results. The simpler models, despite their limitations, are time and cost effective and yield considerable outcomes even on the smaller datasets.

Evolution of Surgical Robot Systems Enhanced by Artificial Intelligence: A Review

“…Several related reviews share similar perspectives, offering compelling evidence indicating the superiority of AI techniques over conventional methods in surgical procedures. [173,174] AI-aided SRS surpasses traditional and robot-only surgeries, offering a safer and more efficient surgical system with fewer or no complications. [175] The extensive application of AI is expected to bolster human performance.…”

“…Several related reviews share similar perspectives, offering compelling evidence indicating the superiority of AI techniques over conventional methods in surgical procedures. [ 173,174 ]…”

A surgical activity model of laparoscopic cholecystectomy for co-operation with collaborative robots

“…LC is a popular benchmark procedure for surgical workflow recognition with ML due to it being a high-volume procedure performed by novice surgeons [21]. However, more complex surgical procedures performed by experienced surgeons such as robot-assisted minimally invasive esophagectomy (RAMIE) are becoming increasingly relevant for surgical workflow recognition due to higher process standardization and clinical need for assistance systems to further improve patient outcomes [56,57].…”

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

“…Given the differences in application, reviews of simulation training for general robotics, such as [47], focus on the use of RGB(-D) imaging, which generally does not apply in the context of MIS. Further, although previous reviews include recent advances in MIS [48][49][50][51][52][53][54][55][56][57][58][59], robotic-assisted MIS [55,[60][61][62][63], machine learning in surgical interventions [34,35,[64][65][66][67][68][69][70][71][72], or surgical simulation for human training purposes [73][74][75][76], in silico training specifically for intelligent MIS systems remains an emerging area deserving of an introduction. We focus this review on frameworks and successful applications in three imaging modalities which have received the bulk of researchers' attention, namely endoscopy, ultrasound (US), and x-ray.…”