Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

Abstract

identifying image features that are robust with respect to segmentation variability is a tough challenge in radiomics. So far, this problem has mainly been tackled in test-retest analyses. in this work we analyse radiomics feature reproducibility in two phases: first with manual segmentations provided by four expert readers and second with probabilistic automated segmentations using a recently developed neural network (pHiseg). We test feature reproducibility on three publicly available datasets of lung, kidney and liver lesions. We find consistent results both over manual and automated segmentations in all three datasets and show that there are subsets of radiomic features which are robust against segmentation variability and other radiomic features which are prone to poor reproducibility under differing segmentations. By providing a detailed analysis of robustness of the most common radiomics features across several datasets, we envision that more reliable and reproducible radiomic models can be built in the future based on this work.

Combining radiomic phenotypes of non-small cell lung cancer with liquid biopsy data may improve prediction of response to EGFR inhibitors

“…In addition, we used manual segmentation of tumors by only one human expert. While studies have shown that in general tumor segmentation and radiomic feature extraction could be affected by inter-rater variability 45 , recent studies suggest that such variability may not necessarily affect the robustness of all radiomic features 46 . In a preliminary evaluation, we also recently showed that despite inter-reader variation, radiomic features extracted from segmentations obtained by different human raters tend to be highly correlated and have similar predictive value 47 .…”

A multiparametric approach to improve the prediction of response to immunotherapy in patients with metastatic NSCLC

“…However, it is not clear to what degree segmentation variability has an impact on radiomics features, even considering that a universal automatic segmentation algorithm has not been validated and established for all image applications, and some features may not show stability and reproducibility using different methods. Furthermore, automatic segmentation means “probabilistic” segmentation, and the ground truth for automatic boundering comes only for big datasets able to train the neural network [ 61 ]. Unfortunately, 11 patients were not sufficient to apply an automatic or semi-automatic approach, leading to more variable results than manual segmentation.…”

Deep learning-based automatic meibomian gland segmentation and morphology assessment in infrared meibography

“…To determine the variability and repeatability (test–retest reliability) 28 , 29 of the developed algorithm, we additionally have compared two manual annotations by the same observer with two automatic segmentations, using the same test dataset of 100 images (50 upper and 50 lower eyelids). The same observer (M.A.K.S.)…”