VR-Caps: A Virtual Environment for Capsule Endoscopy

İncetan, Kağan; Celik, Ibrahim Omer; Obeid, Abdulhamid; Gokceler, Guliz Irem; Ozyoruk, Kutsev Bengisu; Almalioglu, Yasin; Chen, Richard J.; Mahmood, Faisal; Gilbert, Hunter B.; Durr, Nicholas J.; Turan, Mehmet

doi:10.1016/j.media.2021.101990

Cited by 52 publications

(32 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also show results in Fig. 3 for 11 real OC video sequences from Ma et al [12] and 2 from recently-released VR-CAPS simulator [9]. For both, FoldIt and XDCycleGAN, we show an overlay view, where the fold segmentation is extracted from the network output and superimposed in blue on the OC input.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Mathew¹,

Nadeem²,

Kaufman³

2021

Preprint

View full text Add to dashboard Cite

Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Fig.3. Results of FoldIt (ours) and XDCycleGAN model[15] are shown on video sequences from Ma et al[12] and recently released VR-CAPS colon simulator[9]. Complete video sequences are provided in the supplementary video.…”

mentioning

confidence: 99%

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Mathew¹,

Nadeem²,

Kaufman³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead of using data-driven approaches such as generative modelling, algorithms that explicitly model physical properties (for example, light scattering in tissue) can be used to generate biologically accurate synthetic data 20,62,63 . For complicated medical procedures such as colonoscopies, virtual environments akin to those used to develop self-driving cars could be used to train AI-based capsule endoscopes to navigate the gastrointestinal tract 64 . Unlike synthetic data from generative models, simulation-based synthetic data from forward models are created from existing clinical reference standards, medical prior knowledge and physical laws.…”

Section: Algorithms Grounded On Real Datamentioning

confidence: 99%

Synthetic data in machine learning for medicine and healthcare

et al. 2021

Self Cite

View full text Add to dashboard Cite

“…The authors have introduced the new virtual reality capsule to simulate and identify the normal and abnormal regions. This environment is generated new 3D images for gastrointestinal diseases [ 24 ]. Local spatial features are retrieved from pixels of interest in a WCE image using a linear separation approach in this paper.…”

Section: Related Workmentioning

confidence: 99%

Gastrointestinal Tract Disease Classification from Wireless Endoscopy Images Using Pretrained Deep Learning Model

Yogapriya¹,

Sumithra

Anitha³

et al. 2021

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

Wireless capsule endoscopy is a noninvasive wireless imaging technology that becomes increasingly popular in recent years. One of the major drawbacks of this technology is that it generates a large number of photos that must be analyzed by medical personnel, which takes time. Various research groups have proposed different image processing and machine learning techniques to classify gastrointestinal tract diseases in recent years. Traditional image processing algorithms and a data augmentation technique are combined with an adjusted pretrained deep convolutional neural network to classify diseases in the gastrointestinal tract from wireless endoscopy images in this research. We take advantage of pretrained models VGG16, ResNet-18, and GoogLeNet, a convolutional neural network (CNN) model with adjusted fully connected and output layers. The proposed models are validated with a dataset consisting of 6702 images of 8 classes. The VGG16 model achieved the highest results with 96.33% accuracy, 96.37% recall, 96.5% precision, and 96.5% F1-measure. Compared to other state-of-the-art models, the VGG16 model has the highest Matthews Correlation Coefficient value of 0.95 and Cohen’s kappa score of 0.96.

show abstract

VR-Caps: A Virtual Environment for Capsule Endoscopy

Cited by 52 publications

References 31 publications

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Synthetic data in machine learning for medicine and healthcare

Gastrointestinal Tract Disease Classification from Wireless Endoscopy Images Using Pretrained Deep Learning Model

Contact Info

Product

Resources

About