HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy

Borgli, Hanna; Thambawita, Vajira; Smedsrud, Pia H.; Hicks, Steven A.; Jha, Debesh; Eskeland, Sigrun Losada; Randel, Kristin Ranheim; Pogorelov, Konstantin; Lux, Mathias; Nguyen, Duc Tien Dang; Johansen, Dag; Griwodz, Carsten; Stensland, Håkon Kvale; Garcia-Ceja, Enrique; Schmidt, Peter T.; Hammer, Hugo Lewi; Riegler, Michael; Halvorsen, Pål; Lange, Thomas de

doi:10.1038/s41597-020-00622-y

Cited by 278 publications

(188 citation statements)

References 60 publications

Supporting

Mentioning

187

Contrasting

Order By: Relevance

“…24 Hickory and Laurel are recently concluded randomized, double-blind, placebo-controlled, phase-III studies, involving 609 and 359 patients with UC, respectively. Hickory evaluated the safety, efficacy, and tolerability of etrolizumab during B Gutierrez Becker, F Arcadu et al In addition to the data obtained from the clinical trials, we externally validated the performance of our models using still frames obtained from the publicly available Hyperkvasir data set, 25 which are equipped with an MCES score assigned by the authors of the data set.…”

Section: Data Setmentioning

confidence: 99%

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data

Becker

Arcadu

Thalhammer

et al. 2021

Clin Med Insights Gastroenterol

View full text Add to dashboard Cite

Introduction: The Mayo Clinic Endoscopic Subscore is a commonly used grading system to assess the severity of ulcerative colitis. Correctly grading colonoscopies using the Mayo Clinic Endoscopic Subscore is a challenging task, with suboptimal rates of interrater and intrarater variability observed even among experienced and sufficiently trained experts. In recent years, several machine learning algorithms have been proposed in an effort to improve the standardization and reproducibility of Mayo Clinic Endoscopic Subscore grading. Methods: Here we propose an end-to-end fully automated system based on deep learning to predict a binary version of the Mayo Clinic Endoscopic Subscore directly from raw colonoscopy videos. Differently from previous studies, the proposed method mimics the assessment done in practice by a gastroenterologist, that is, traversing the whole colonoscopy video, identifying visually informative regions and computing an overall Mayo Clinic Endoscopic Subscore. The proposed deep learning–based system has been trained and deployed on raw colonoscopies using Mayo Clinic Endoscopic Subscore ground truth provided only at the colon section level, without manually selecting frames driving the severity scoring of ulcerative colitis. Results and Conclusion: Our evaluation on 1672 endoscopic videos obtained from a multisite data set obtained from the etrolizumab Phase II Eucalyptus and Phase III Hickory and Laurel clinical trials, show that our proposed methodology can grade endoscopic videos with a high degree of accuracy and robustness (Area Under the Receiver Operating Characteristic Curve = 0.84 for Mayo Clinic Endoscopic Subscore ⩾ 1, 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 2 and 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 3) and reduced amounts of manual annotation. Plain language summary Patient, caregiver and provider thoughts on educational materials about prescribing and medication safety Artificial intelligence can be used to automatically assess full endoscopic videos and estimate the severity of ulcerative colitis. In this work, we present an artificial intelligence algorithm for the automatic grading of ulcerative colitis in full endoscopic videos. Our artificial intelligence models were trained and evaluated on a large and diverse set of colonoscopy videos obtained from concluded clinical trials. We demonstrate not only that artificial intelligence is able to accurately grade full endoscopic videos, but also that using diverse data sets obtained from multiple sites is critical to train robust AI models that could potentially be deployed on real-world data.

show abstract

Section: Data Setmentioning

confidence: 99%

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data

Becker

Arcadu

Thalhammer

et al. 2021

Clin Med Insights Gastroenterol

View full text Add to dashboard Cite

show abstract

“…The second dataset is called HyperKvasir (Borgli et al, 2020) and named Dataset II. The images and videos of this dataset were acquired using standard endoscopy equipment from Olympus (Olympus Europe, Hamburg, Germany) and Pentax (Pentax Medical Europe, Hamburg, Germany) at a Norwegian hospital from 2008 to 2016.…”

Section: Dataset Descriptionmentioning

confidence: 99%

GASTRO-CADx: a three stages framework for diagnosing gastrointestinal diseases

Attallah¹,

Sharkas²

2021

PeerJ Computer Science

View full text Add to dashboard Cite

Gastrointestinal (GI) diseases are common illnesses that affect the GI tract. Diagnosing these GI diseases is quite expensive, complicated, and challenging. A computer-aided diagnosis (CADx) system based on deep learning (DL) techniques could considerably lower the examination cost processes and increase the speed and quality of diagnosis. Therefore, this article proposes a CADx system called Gastro-CADx to classify several GI diseases using DL techniques. Gastro-CADx involves three progressive stages. Initially, four different CNNs are used as feature extractors to extract spatial features. Most of the related work based on DL approaches extracted spatial features only. However, in the following phase of Gastro-CADx, features extracted in the first stage are applied to the discrete wavelet transform (DWT) and the discrete cosine transform (DCT). DCT and DWT are used to extract temporal-frequency and spatial-frequency features. Additionally, a feature reduction procedure is performed in this stage. Finally, in the third stage of the Gastro-CADx, several combinations of features are fused in a concatenated manner to inspect the effect of feature combination on the output results of the CADx and select the best-fused feature set. Two datasets referred to as Dataset I and II are utilized to evaluate the performance of Gastro-CADx. Results indicated that Gastro-CADx has achieved an accuracy of 97.3% and 99.7% for Dataset I and II respectively. The results were compared with recent related works. The comparison showed that the proposed approach is capable of classifying GI diseases with higher accuracy compared to other work. Thus, it can be used to reduce medical complications, death-rates, in addition to the cost of treatment. It can also help gastroenterologists in producing more accurate diagnosis while lowering inspection time.

show abstract

“…CVAT, like CdV, allows videos to be imported and annotated frame-by-frame (Figure 1). Using a subsample of polyp videos from the Hyper-Kvasir dataset [7], five independent annotators were asked to draw bounding boxes around polyps identified in videos from the dataset. A test set of 25,744 frames was used.…”

Section: A Comparative Study Was Conducted For CDV To the Popular Opementioning

confidence: 99%

Novel artificial intelligence-driven software significantly shortens the time required for annotation in computer vision projects

Hansen

Landau

Patel

et al. 2020

Preprint

View full text Add to dashboard Cite

The contribution of artificial intelligence (AI) to endoscopy is rapidly expanding. Accurate labelling of source data (video frames) remains the rate-limiting step for such projects and is a painstaking, cost-inefficient, time-consuming process. A novel software platform, Cord Vision (CdV) allows automated annotation based on 'embedded intelligence'. The user manually labels a representative proportion of frames in a section of video (typically 5%), to create 'micro-models' which allow accurate propagation of the label throughout the remaining video frames. This could drastically reduce the time required for annotation. We conducted a comparative study with an open-source labelling platform (CVAT) to determine speed and accuracy of labelling. Across 5 users, CdV resulted in a significant increase in labelling performance (p<0.0005) compared to CVAT with micro-models achieving >97% accuracy for bounding box placement. This advance represents a valuable first step in AI-image analysis projects.

show abstract

HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy

Cited by 278 publications

References 60 publications

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data

GASTRO-CADx: a three stages framework for diagnosing gastrointestinal diseases

Novel artificial intelligence-driven software significantly shortens the time required for annotation in computer vision projects

Contact Info

Product

Resources

About