The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Fu, Zuoming; Jin, Ziyi; Zhang, Chongan; He, Zhongyu; Zha, Zhenzhou; Hu, Chunyong; Gan, Tianyuan; Yan, Qinglai; Wang, Peng; Ye, Xuesong

doi:10.1109/access.2021.3065104

Cited by 39 publications

(16 citation statements)

References 200 publications

(207 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, Minimally Invasive Surgery (MIS) is a type of endoscopic procedure that is increasingly becoming a mainstream medical procedure, overtaking traditional surgical operations and treatments. Such endoscopic intervention entails a less traumatic experience and less pain for the patient, quicker recovery after surgery and shortened hospital stays [9].…”

Section: Medical Contextmentioning

confidence: 99%

“…In recent years computer vision (CV) has been playing significant role in endoscopic explorations aided by novel deep learning (DL) techniques, which give computers the visual and temporal learning abilities to understand complex surgical procedures in hollow organs [9]. The integration of such techniques is currently being investigated also for laparoscopy and MIS applications (i.e., such as instrument tracking, endoscopic view enhancement and suspicious lesion tracking [9]). Among these applications, 3D surface reconstruction seems to be a promising solution for the poor depth perception which prevents the full-fledged adoption of the above-mentioned techniques in MIS and Robotic-assisted surgery (RAS).…”

Section: Motivation For Our Proposalmentioning

confidence: 99%

“…Nonetheless, another requirement for developing usable IE methods in endoscopy is related their inference time or latency. Most of the works found in the literature take several seconds to enhance a single image, hampering their use in real-time image processing applications such as video processing of underwater [12], X-ray imaging [14] and restoration in video endoscopy [9] . Endoscopy is an application area which requires dynamic and real-time image enhancement, which additionally needs to be accurate in the sense of not introducing any artifacts and we believe that the work presented in [1] can serve a launching pad to satisfy these requirements.…”

Section: State Of the Artmentioning

confidence: 99%

See 2 more Smart Citations

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Garcia-Vega¹,

Espinosa²,

Ochoa-Ruiz³

et al. 2022

Preprint

View full text Add to dashboard Cite

Endoscopy is the most widely used medical technique for cancer and polyp detection inside hollow organs. However, images acquired by an endoscope are frequently affected by illumination artefacts due to the enlightenment source orientation. There exist two major issues when the endoscope's light source pose suddenly changes: overexposed and underexposed tissue areas are produced. These two scenarios can result in misdiagnosis due to the lack of information in the affected zones or hamper the performance of various computer vision methods (e.g., SLAM, structure from motion, optical flow) used during the non invasive examination. The aim of this work is two-fold: i) to introduce a new synthetically generated data-set generated by a generative adversarial techniques and ii) and to explore both shallow based and deep learning-based image-enhancement methods in overexposed and underexposed lighting conditions. Best quantitative results (i.e., metric based results), were obtained by the deep-learnnig -based LMSPEC method, besides a running time around 7.6 fps.

show abstract

Section: Medical Contextmentioning

confidence: 99%

Section: Motivation For Our Proposalmentioning

confidence: 99%

Section: State Of the Artmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Garcia-Vega¹,

Espinosa²,

Ochoa-Ruiz³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The surgeon (or his assistant) may navigate the endoscope in order to change the viewpoint, but any advanced operations on the image are usually performed later, using appropriate computer programs. Contemporary image processing systems offer a very rich spectrum of possible options: from simple zooming on the area of interest to such advanced techniques as instrument tracking, suspicious lesion tracking, or endoscopic view expansion based on 3D surface reconstruction [ 32 ]. According to Boskoski and Costamagna [ 33 ], the future applications and developments of endoscopy robots will include high-resolution imaging, 3D vision, and advanced optical analysis.…”

Section: Introductionmentioning

confidence: 99%

Scenario-Based Programming of Voice-Controlled Medical Robotic Systems

Rogowski

2022

Sensors

View full text Add to dashboard Cite

An important issue in medical robotics is communication between physicians and robots. Speech-based communication is of particular advantage in robot-assisted surgery. It frees the surgeon’s hands; hence, he can focus on the principal tasks. Man-machine voice communication is the subject of research in various domains (industry, social robotics), but medical robots are very specific. They must precisely synchronize their activities with operators. Voice commands must be possibly short. They must be executed without significant delays. An important factor is the use of a vision system that provides visual information in direct synchronization with surgeon actions. Its functions could be also controlled using speech. The aim of the research presented in this paper was to develop a method facilitating creation of voice-controlled medical robotic systems, fulfilling the mentioned requirements and taking into account possible scenarios of man-machine collaboration in such systems. A robot skill description (RSD) format was proposed in order to facilitate programming of voice control applications. A sample application was developed, and experiments were conducted in order to draw conclusions regarding the usefulness of speech-based interfaces in medical robotics. The results show that a reasonable selection of system functions controlled by voice may lead to significant improvement of man-machine collaboration.

show abstract

“…One of the main forms of assistance is providing accurate tracking of medical instruments using computer vision techniques, such as object detection and localisation or instance segmentation methods Ward et al (2021). These systems are expected to be a crucial component in tasks rang-ing from surgical navigation, skill analysis and complication prediction during surgeries, as well as other computer integrated surgery (CIS) applications Maier-Hein et al (2021b); Fu et al (2021). Nonetheless, methods for accurate tracking of instruments are often deployed in difficult operational scenarios in which the presence of bleeding, over or under exposed frames, smoke, reflection and other types of artifacts are oftentimes unavoidable Bodenstedt et al (2018).…”

Section: Introductionmentioning

confidence: 99%

Real-time Instance Segmentation of Surgical Instruments using Attention and Multi-scale Feature Fusion

Angeles-Ceron¹,

Ochoa-Ruiz²,

Chang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Precise instrument segmentation aid surgeons to navigate the body more easily and increase patient safety. While accurate tracking of surgical instruments in real-time plays a crucial role in minimally invasive computer-assisted surgeries, it is a challenging task to achieve, mainly due to 1) complex surgical environment, and 2) model design with both optimal accuracy and speed. Deep learning gives us the opportunity to learn complex environment from large surgery scene environments and placements of these instruments in real world scenarios. The Robust Medical Instrument Segmentation 2019 challenge (ROBUST-MIS) provides more than 10,000 frames with surgical tools in different clinical settings. In this paper, we use a light-weight single stage instance segmentation model complemented with a convolutional block attention module for achieving both faster and accurate inference. We further improve accuracy through data augmentation and optimal anchor localisation strategies. To our knowledge, this is the first work that explicitly focuses on both real-time performance and improved accuracy. Our approach out-performed top team performances in the ROBUST-MIS challenge with over 44% improvement on area-based multi-instance dice metric MI_DSC and 39% on distance-based multi-instance normalized surface dice MI_NSD. We also demonstrate real-time performance (> 60 frames-per-second) with different but competitive variants of our final approach.

show abstract

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Cited by 39 publications

References 200 publications

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Scenario-Based Programming of Voice-Controlled Medical Robotic Systems

Real-time Instance Segmentation of Surgical Instruments using Attention and Multi-scale Feature Fusion

Contact Info

Product

Resources

About