Unsupervised Neural Network for Homography Estimation in Capsule Endoscopy Frames

Gomes, Sara; Valério, Maria Teresa; Salgado, Marta; Oliveira, Hélder P.; Cunha, António

doi:10.1016/j.procs.2019.12.226

Cited by 10 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gomes developed an unsupervised homography evaluation method in the capsule endoscope framework and then applied it to the capsule positioning system. The network can evaluate the homography between two images [ 1 ]. Leenhardt has developed a computer-aided diagnostic tool to detect vasodilation.…”

Section: Introductionmentioning

confidence: 99%

Image Aided Recognition of Wireless Capsule Endoscope Based on the Neural Network

2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Wireless capsule endoscopy is an important method for diagnosing small bowel diseases, but it will collect thousands of endoscopy images that need to be diagnosed. The analysis of these images requires a huge workload and may cause manual reading errors. This article attempts to use neural networks instead of artificial endoscopic image analysis to assist doctors in diagnosing and treating endoscopic images. First, in image preprocessing, the image is converted from RGB color mode to lab color mode, texture features are extracted for network training, and finally, the accuracy of the algorithm is verified. After inputting the retained endoscopic image verification set into the neural network algorithm, the conclusion is that the accuracy of the neural network model constructed in this study is 97.69%, which can effectively distinguish normal, benign lesions, and malignant tumors. Experimental studies have proved that the neural network algorithm can effectively assist the endoscopist’s diagnosis and improve the diagnosis efficiency. This research hopes to provide a reference for the application of neural network algorithms in the field of endoscopic images.

show abstract

Section: Introductionmentioning

confidence: 99%

Image Aided Recognition of Wireless Capsule Endoscope Based on the Neural Network

2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

show abstract

“…videos. It was first introduced in (Nguyen et al 2018 ), and got applied to endoscopy in (Gomes et al 2019 ). The loss in image space, however, can’t account for object motion, and only static scenes are considered in their works.…”

Section: Related Workmentioning

confidence: 99%

Deep homography estimation in dynamic surgical scenes for laparoscopic camera motion extraction

Huber

Ourselin

Bergeles

et al. 2022

Computer Methods in Biomechanics and Biomedical Engineering: Im

View full text Add to dashboard Cite

Current laparoscopic camera motion automation relies on rule-based approaches or only focuses on surgical tools. Imitation Learning (IL) methods could alleviate these shortcomings, but have so far been applied to oversimplified setups. Instead of extracting actions from oversimplified setups, in this work we introduce a method that allows to extract a laparoscope holder's actions from videos of laparoscopic interventions. We synthetically add camera motion to a newly acquired dataset of camera motion free da Vinci surgery image sequences through a novel homography generation algorithm. The synthetic camera motion serves as a supervisory signal for camera motion estimation that is invariant to object and tool motion. We perform an extensive evaluation of state-of-the-art (SOTA) Deep Neural Networks (DNNs) across multiple compute regimes, finding our method transfers from our camera motion free da Vinci surgery dataset to videos of laparoscopic interventions, outperforming classical homography estimation approaches in both, precision by 41%, and runtime on a CPU by 43%.

show abstract

“…On the other hand, a lot of image data is needed for training and the processing is computationally expensive. The process for generating synthetic training data for DH-NN by applying random perspective transformations described in DeTone et al 22 was transferred to capsule endoscopy and thus used in the clinical context 23 . That method was applied to image sequences and developed further for the continuous generation of synthetic data during training by Huber et al 24 .…”

Section: Introductionmentioning

confidence: 99%

Comparison of image registration methods for combining laparoscopic video and spectral image data

Köhler

Pfahl

Moulla

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Laparoscopic procedures can be assisted by intraoperative modalities, such as quantitative perfusion imaging based on fluorescence or hyperspectral data. If these modalities are not available at video frame rate, fast image registration is needed for the visualization in augmented reality. Three feature-based algorithms and one pre-trained deep homography neural network (DH-NN) were tested for single and multi-homography estimation. Fine-tuning was used to bridge the domain gap of the DH-NN for non-rigid registration of laparoscopic images. The methods were validated on two datasets: an open-source record of 750 manually annotated laparoscopic images, presented in this work, and in-vivo data from a novel laparoscopic hyperspectral imaging system. All feature-based single homography methods outperformed the fine-tuned DH-NN in terms of reprojection error, Structural Similarity Index Measure, and processing time. The feature detector and descriptor ORB1000 enabled video-rate registration of laparoscopic images on standard hardware with submillimeter accuracy.

show abstract

Unsupervised Neural Network for Homography Estimation in Capsule Endoscopy Frames

Cited by 10 publications

References 10 publications

Image Aided Recognition of Wireless Capsule Endoscope Based on the Neural Network

Image Aided Recognition of Wireless Capsule Endoscope Based on the Neural Network

Deep homography estimation in dynamic surgical scenes for laparoscopic camera motion extraction

Comparison of image registration methods for combining laparoscopic video and spectral image data

Contact Info

Product

Resources

About