“Real-world” radiomics from multi-vendor MRI: an original retrospective study on the prediction of nodal status and disease survival in breast cancer, as an exemplar to promote discussion of the wider issues

Abstract

Background Most MRI radiomics studies to date, even multi-centre ones, have used “pure” datasets deliberately accrued from single-vendor, single-field-strength scanners. This does not reflect aspirations for the ultimate generalisability of AI models. We therefore investigated the development of a radiomics signature from heterogeneous data originating on six different imaging platforms, for a breast cancer exemplar, in order to provide input into future discussions of the viability of radiomics in “real-world” scenarios where image data are not controlled by specific trial protocols but reflective of routine clinical practice. Methods One hundred fifty-six patients with pathologically proven breast cancer underwent multi-contrast MRI prior to neoadjuvant chemotherapy and/or surgery. From these, 92 patients were identified for whom T2-weighted, diffusion-weighted and contrast-enhanced T1-weighted sequences were available, as well as key clinicopathological variables. Regions-of-interest were drawn on the above image types and, from these, semantic and calculated radiomics features were derived. Classification models using a variety of methods, both with and without recursive feature elimination, were developed to predict pathological nodal status. Separately, we applied the same methods to analyse the information carried by the radiomic features regarding the originating scanner type and field strength. Repeated, ten-fold cross-validation was employed to verify the results. In parallel work, survival modelling was performed using random survival forests. Results Prediction of nodal status yielded mean cross-validated AUC values of 0.735 ± 0.15 (SD) for clinical variables alone, 0.673 ± 0.16 (SD) for radiomic features only, and 0.764 ± 0.16 (SD) for radiomics and clinical features together. Prediction of scanner platform from the radiomics features yielded extremely high values of AUC between 0.91 and 1 for the different classes examined indicating the presence of confounding features for the nodal status classification task. Survival analysis, gave out-of-bag prediction errors of 19.3% (clinical features only), 36.9–51.8% (radiomic features from different combinations of image contrasts), and 26.7–35.6% (clinical plus radiomics features). Conclusions Radiomic classification models whose predictive ability was consistent with previous single-vendor, single-field strength studies have been obtained from multi-vendor, multi-field-strength data, despite clear confounding information being present. However, our sample size was too small to obtain useful survival modelling results.

Wavelet transformation can enhance computed tomography texture features: a multicenter radiomics study for grade assessment of COVID-19 pulmonary lesions

“…Moreover, in addition to building the diagnostic model, this study has also found some interesting insights into CT radiomics. The realization and promotion of AI are seriously affected by the multivendor nature of real-world data, which contain confounding and discrepant information (23). Under this circumstance, the training of deep learning models is often difficult, resulting in underperformance, and the features of engineering-based radiomics may show a superior performance (44,45).…”

“…In addition to detecting lesions, assessing the grade of COVID-19 pulmonary lesions is important for the hierarchical management and treatment of infected patients (19). However, few studies have focused on the use of AI in grading pulmonary lesions (20), and those that exist have only used "pure" datasets accrued from unified vendor scanners (21,22), which limits the real generalizability of their AI models (23). Thus, it is necessary to develop a more robust AI grading tool based on real-world multicenter datasets.…”

Prediction of Incomplete Response of Primary Tumour Based on Clinical and Radiomics Features in Inoperable Head and Neck Cancers after Definitive Treatment

“…In the spirit of leveraging real-world data, similarly to the recent study in breast cancer [ 25 ], we attempted to use not only the standard, non-contrast enhanced CT images acquired in the treatment planning process, but also a heterogenous set of contrast-enhanced CT diagnostic images obtained in our center and other hospitals shortly before treatment.…”

“…Thanks to significantly higher availability, using these diagnostic images could help to build larger data sets, provided that the balance between quality and quantity is maintained. It would also help to improve the robustness and generalization of radiomics models, supporting their wider adoption and use [ 25 , 26 ]. Unfortunately, real-world contrast enhanced CT images collected in this study were too heterogeneous to build a predictive radiomics model with satisfactory performance, enabling further clinical testing and use.…”

A Low-Dose CT-Based Radiomic Model to Improve Characterization and Screening Recall Intervals of Indeterminate Prevalent Pulmonary Nodules

“…First, the retrospective design might be prone to some confounding factors, including the selection of some patients. Indeed, the deployment of radiomics to “real-world” scenarios, where imaging data are not controlled by specific trial protocols but retrieved from the current clinical practice, is still challenging to implement [ 59 ]. In particular, in compliance with this very recent study, a nested cross-validation was adopted, along with hyper-parameter tuning, to avoid over-optimistic results despite our single-center study.…”