Improving radiomic model reliability using robust features from perturbations for head-and-neck carcinoma

Abstract

BackgroundUsing high robust radiomic features in modeling is recommended, yet its impact on radiomic model is unclear. This study evaluated the radiomic model’s robustness and generalizability after screening out low-robust features before radiomic modeling. The results were validated with four datasets and two clinically relevant tasks.Materials and methodsA total of 1,419 head-and-neck cancer patients’ computed tomography images, gross tumor volume segmentation, and clinically relevant outcomes (distant metastasis and local-regional recurrence) were collected from four publicly available datasets. The perturbation method was implemented to simulate images, and the radiomic feature robustness was quantified using intra-class correlation of coefficient (ICC). Three radiomic models were built using all features (ICC > 0), good-robust features (ICC > 0.75), and excellent-robust features (ICC > 0.95), respectively. A filter-based feature selection and Ridge classification method were used to construct the radiomic models. Model performance was assessed with both robustness and generalizability. The robustness of the model was evaluated by the ICC, and the generalizability of the model was quantified by the train-test difference of Area Under the Receiver Operating Characteristic Curve (AUC).ResultsThe average model robustness ICC improved significantly from 0.65 to 0.78 (P< 0.0001) using good-robust features and to 0.91 (P< 0.0001) using excellent-robust features. Model generalizability also showed a substantial increase, as a closer gap between training and testing AUC was observed where the mean train-test AUC difference was reduced from 0.21 to 0.18 (P< 0.001) in good-robust features and to 0.12 (P< 0.0001) in excellent-robust features. Furthermore, good-robust features yielded the best average AUC in the unseen datasets of 0.58 (P< 0.001) over four datasets and clinical outcomes.ConclusionsIncluding robust only features in radiomic modeling significantly improves model robustness and generalizability in unseen datasets. Yet, the robustness of radiomic model has to be verified despite building with robust radiomic features, and tightly restricted feature robustness may prevent the optimal model performance in the unseen dataset as it may lower the discrimination power of the model.

Multi-omics and Multi-VOIs to predict esophageal fistula in esophageal cancer patients treated with radiotherapy

“…The use of homogeneous IMRT treatment may undermine the generalizability of the results, potentially impacting their overall quality. Third, several studies have analyzed feature repeatability and reproducibility by considering the impact of VOI segmentation and CT image acquisition (Teng et al 2022a , b ; Placidi et al 2020 ; Zwanenburg et al 2019 ; Larue et al 2017 ; Lafata et al 2018 ). It is worth comprehensively studying models that use robust and repeatable features before clinical use.…”

Multi-omics and Multi-VOIs to Predict Esophageal Fistula in Esophageal Cancer Patients Treated with Radiotherapy

“…The investigation should be reanalyzed for other radiotherapy techniques, such as Volume Modulated Radiation Therapy (VMAT) or a combination of IMRT and VMAT owing to the difference in dose distribution. Thirdly, several studies have analyzed feature repeatability and reproducibility by considering the impact of VOI segmentation and CT image acquisition [43][44][45][46][47][48]. It is worth comprehensively studying models that use robust and repeatable features before clinical use.…”

Artificial intelligence-driven radiomics study in cancer: the role of feature engineering and modeling

“…Reproducibility is a measure of the variability of repeated measurements of the same or similar quantitative imaging biomarkers in a real clinical environment and is affected by external factors that cannot be strictly controlled, such as operators, measurement systems, and measurement equipment [ 86 , 87 ]. Thus, reproducibility represents stability, so radiomics studies must ensure that the radiomics features they use have high reproducibility, such that their models generate similar classification results in different clinical environments.…”