Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review

Abstract

Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, postprocessing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisitioninduced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review.More future researches on radiomics reliability are warranted.

Acquisition repeatability of MRI radiomics features in the head and neck: a dual-3D-sequence multi-scan study

“…In recent years, an increasing number of studies have reported promising preliminary results on the use of radiomics for the diagnosis and prognosis of HNC [ 17 – 21 ]. However, the broad reproducibility, validity, and generality of radiomics remain open to question or challenge owing to the wide variety of confounding factors that could substantially impact every procedure of the complicated radiomics workflow and lead to uncertainty, instability, or unreliability of the results of radiomics analysis [ 22 – 24 ]. To date, the accumulated evidence remains insufficient to justify the deployment of radiomics in routine clinical practice.…”

“…This study specifically addressed an important source of MRI radiomics feature unreliability due to multiple acquisitions, but did not assess the influence of many other confounding factors on feature values, such as image reconstruction, segmentation, and other image post-processing methods [ 49 ]. Although image acquisition has been found to be more impactful on feature reliability than other sources in MRI radiomics [ 24 ], it is equally important to investigate the impact of other factors on the reliability of radiomics features. Meanwhile, only a small subset of first-order and texture original radiomics features was included for analysis in this study among the thousands of radiomics features in the original and transformed images proposed for radiomics modeling in the medical literature.…”

“…In particular, shape features indicating the geometric characteristics of various tissues (such as size or volume) are conventionally used as imaging markers in cancer staging [ 50 ] and treatment response evaluation [ 51 ], but were not included in the present work. Although features in the transformed domains might provide more candidates for radiomics modeling and different information on tissue properties, it has been found that image transformation does not necessarily improve radiomics features reliability [ 24 ]. Therefore, the absence of transformed features in this study may not seriously compromise the validity and interpretation of the results.…”

“…Radiomics [ 14 – 16 ] has attracted increasing research interest as a multidimensional data mining technique in medical imaging for the diagnosis and prognosis of HNC in recent years [ 17 – 21 ]. Despite the promising results reported in these studies, the reliability of radiomics features remains a major obstacle to the broad validity and generality of radiomics in routine clinical use [ 22 – 24 ].…”

Robustness Of Radiomics To Variations In Segmentation Methods In Multimodal Brain MRI

“…Reliability of radiomics with respect to image acquisition and image processing has been under extensive scienti c scrutiny, especially in oncology, where 87% of studies on robustness of radiomic features has been performed. 43 This research has yielded standardization of MRI-acquisition protocols and standardization of radiomics de nitions that the eld of psychoradiology can draw from. 44,45 However, the segmentation in oncology has mainly been focused on effects of semi-automatic segmentation 34,46−51 of tumor mass rating, whereas psychoradiology makes use of different fully-automatic whole-brain anatomical segmentation methods.…”