RIL-Contour: a Medical Imaging Dataset Annotation Tool for and with Deep Learning

Abstract

Deep-learning algorithms typically fall within the domain of supervised artificial intelligence and are designed to “learn” from annotated data. Deep-learning models require large, diverse training datasets for optimal model convergence. The effort to curate these datasets is widely regarded as a barrier to the development of deep-learning systems. We developed RIL-Contour to accelerate medical image annotation for and with deep-learning. A major goal driving the development of the software was to create an environment which enables clinically oriented users to utilize deep-learning models to rapidly annotate medical imaging. RIL-Contour supports using fully automated deep-learning methods, semi-automated methods, and manual methods to annotate medical imaging with voxel and/or text annotations. To reduce annotation error, RIL-Contour promotes the standardization of image annotations across a dataset. RIL-Contour accelerates medical imaging annotation through the process of annotation by iterative deep learning (AID). The underlying concept of AID is to iteratively annotate, train, and utilize deep-learning models during the process of dataset annotation and model development. To enable this, RIL-Contour supports workflows in which multiple-image analysts annotate medical images, radiologists approve the annotations, and data scientists utilize these annotations to train deep-learning models. To automate the feedback loop between data scientists and image analysts, RIL-Contour provides mechanisms to enable data scientists to push deep newly trained deep-learning models to other users of the software. RIL-Contour and the AID methodology accelerate dataset annotation and model development by facilitating rapid collaboration between analysts, radiologists, and engineers.

Complete abdomen and pelvis segmentation using U‐net variant architecture

“…A total of ten analysts participated in this study. Segmentation was performed using RIL‐Contour, an open‐source annotation software 12 . Several techniques were used to expedite this process.…”

“…Segmentation was performed using RIL-Contour, an open-source annotation software. 12 Several techniques were used to expedite this process. Soft-tissue, muscle, fat, and bone segmentations were generated using a previously developed model 18 and were then provided to analysts who corrected existing ROIs and manually segmented novel anatomy.…”

“…Access to high‐quality annotated data remains the single‐biggest limitation to developing deep learning‐based approaches. For this report, 84 CT volumes were segmented by ten image analysts and corrected by four radiologists using an open‐source annotation tool 12 …”

“…For this report, 84 CT volumes were segmented by ten image analysts and corrected by four radiologists using an open-source annotation tool. 12 Although manual segmentation is still widely accepted as the gold standard approach, few results on the accuracy of human tracing are found in the literature. In addition to reporting the accuracy of our model, we report metrics for interobserver variability of multiple tracers using semiautomated segmentation on all abdominal organs, and purely manual segmentation on a subset of organs.…”

Ceramide Dynamics and Prognostic Value in Acute and Subacute Ischemic Stroke: Preliminary Findings in a Clinical Cohort

“…For each patient, manual segmentations of the infarct areas were obtained using the software RIL-Contour [24]. Areas of restricted diffusion were segmented on diffusion-weighted imaging (DWI) images without using a priori thresholds.…”