Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Abstract

Radiomics is the method of choice for investigating the association between cancer imaging phenotype, cancer genotype and clinical outcome prediction in the era of precision medicine. The fast dispersal of this new methodology has benefited from the existing advances of the core technologies involved in radiomics workflow: image acquisition, tumor segmentation, feature extraction and machine learning. However, despite the rapidly increasing body of publications, there is no real clinical use of a developed radiomics signature so far. Reasons are multifaceted. One of the major challenges is the lack of reproducibility and generalizability of the reported radiomics signatures (features and models). Sources of variation exist in each step of the workflow; some are controllable or can be controlled to certain degrees, while others are uncontrollable or even unknown. Insufficient transparency in reporting radiomics studies further prevents translation of the developed radiomics signatures from the bench to the bedside. This review article first addresses sources of variation, which is illustrated using demonstrative examples. Then, it reviews a number of published studies and progresses made to date in the investigation and improvement of feature reproducibility and model performance. Lastly, it discusses potential strategies and practical considerations to reduce feature variability and improve the quality of radiomics study. This review focuses on CT image acquisition, tumor segmentation, quantitative feature extraction, and the disease of lung cancer.

Radiomics for Identification and Prediction in Metastatic Prostate Cancer: A Review of Studies

“…The radiomics workflow contains a myriad of factors that can profoundly affect the resulting quantitative imaging biomarker measurement; anything from the algorithm used to reconstruct the medical image to the interpolation method utilised in an up-or down-sampling procedure can introduce variability into radiomics research (29,30). The sensitivity of extracted features to a host of procedural factors has contributed to poor scientific reproducibility in radiomics research that substantially affects its translational capacity (31). Recognising the need to ensure the scientific rigour of radiomics studies, Lambin et al (32) introduced the Radiomics Quality Score (RQS), a points-based system that rewards and penalises radiomics papers according to specific attributes of their methodologies.…”

Assessment of CT to CBCT contour mapping for radiomic feature analysis in prostate cancer

“…These incongruous results may be due to the differing locations within the body (prostate versus liver) and/or the differing contour sizes, but only further the idea that the robustness of radiomic features should be evaluated as they can vary by location. Future radiomic studies should consider the location specific radiomic feature robustness, as the radiomic feature derived data has varying dependence on contour as seen in this study and others 3 – 5 , 10 – 12 .…”

“…Previous studies have linked several radiomic features directly to patient survival 2 . Research has shown the power of radiomics for many disease sites; however, these studies also show variability with respect to imaging modality, reconstruction algorithms, feature selection, and volume of interest (VOI) 3 – 9 . Several groups have studied the robustness of radiomic features with respect to contouring variability 3 – 5 , 10 – 12 .…”

“…Research has shown the power of radiomics for many disease sites; however, these studies also show variability with respect to imaging modality, reconstruction algorithms, feature selection, and volume of interest (VOI) 3 – 9 . Several groups have studied the robustness of radiomic features with respect to contouring variability 3 – 5 , 10 – 12 . Contours are typically created by a trained radiation oncologist; however, inter-, and intra-observer contouring variation can still be significant when considering radiomics 10 , 13 .…”

Site-Specific Variation in Radiomic Features of Head and Neck Squamous Cell Carcinoma and Its Impact on Machine Learning Models

“…Image quality and the extent of image preprocessing as well as the parameters used during image preprocessing can have a substantial impact on the reliability of radiomic features and the potential of these features capturing tumor heterogeneity [22,23]. These variabilities can lead to a lack of reproducibility and generalizability of the reported radiomic features [24,25]. We also hypothesize that the variation between radiomic features can also be influenced by the tumor site.…”