Multicenter CT phantoms public dataset for radiomics reproducibility tests

Abstract

PurposeThe aim of this paper is to describe a public, open‐access, computed tomography (CT) phantom image set acquired at three centers and collected especially for radiomics reproducibility research. The dataset is useful to test radiomic features reproducibility with respect to various parameters, such as acquisition settings, scanners, and reconstruction algorithms.Acquisition and validation methodsThree phantoms were scanned in three independent institutions. Images of the following phantoms were acquired: Catphan 700 and COPDGene Phantom II (Phantom Laboratory, Greenwich, NY, USA), and the Triple modality 3D Abdominal Phantom (CIRS, Norfolk, VA, USA). Data were collected at three Dutch medical centers: MAASTRO Clinic (Maastricht, NL), Radboud University Medical Center (Nijmegen, NL), and University Medical Center Groningen (Groningen, NL) with scanners from two different manufacturers Siemens Healthcare and Philips Healthcare. The following acquisition parameter were varied in the phantom scans: slice thickness, reconstruction kernel, and tube current.Data format and usage notesWe made the dataset publically available on the Dutch instance of “Extensible Neuroimaging Archive Toolkit‐XNAT” (https://xnat.bmia.nl). The dataset is freely available and reusable with attribution (Creative Commons 3.0 license).Potential applicationsOur goal was to provide a findable, open‐access, annotated, and reusable CT phantom dataset for radiomics reproducibility studies. Reproducibility testing and harmonization are fundamental requirements for wide generalizability of radiomics‐based clinical prediction models. It is highly desirable to include only reproducible features into models, to be more assured of external validity across hitherto unseen contexts. In this view, phantom data from different centers represent a valuable source of information to exclude CT radiomic features that may already be unstable with respect to simplified structures and tightly controlled scan settings. The intended extension of our shared dataset is to include other modalities and phantoms with more realistic lesion simulations.

A review of original articles published in the emerging field of radiomics

“…One of the most serious problems is the reliability of the abundant imaging-derived signatures proposed by radiomics [42]. Reproducibility of radiomics has been demonstrated to be vital for its use in clinical practice and many studies have put effort into testing the reproducibility of radiomics results [43][44][45]. However, in this study, we found that there is currently a lack of authentication on the reliability of radiomics signatures in radiomics research (single-center studies accounted for the majority of studies).…”

Radiogenomics in Colorectal Cancer

“…Researchers should thus be encouraged to feed the existing databases (The Cancer Imaging Archive-TCIA, The Cancer Genome Atlas-TCGA, response to drug or radiation therapy -RIDER). Publicly available phantom datasets, intended for radiomics reproducibility tests, could help assess the influence of acquisition settings in order to eliminate non-robust radiomic features [ 113 ].…”

Radiomics in medical imaging—“how-to” guide and critical reflection

“…Publicly available datasets like the RIDER dataset 10 help to gain knowledge about the impact of varying factors in radiomics [ 121 ]. Also, the availability of a public phantom dataset, intended for radiomics reproducibility tests on CT, could help to further assess the influence of acquisition settings in order to eliminate non-robust radiomic features [ 128 ]. However, studies are needed to show if robustness data acquired on a phantom can be translated to the human.…”