Assessing the reproducibility of CBCT‐derived radiomics features using a novel three‐dimensional printed phantom

Abstract

Purpose Radiomics modeling is an exciting avenue for enhancing clinical decision making and personalized treatment. Radiation oncology patients often undergo routine imaging for position verification, particularly using LINAC‐mounted cone beam computed tomography (CBCT). The wealth of imaging data collected in modern radiation therapy presents an ideal use case for radiomics modeling. Despite this, texture feature (TF) calculation can be limited by concerns over feature stability and reproducibility; in theory, this issue is compounded by the relatively poor image quality of CBCT, as well as variation of acquisition and reconstruction parameters. Methods In this study, we developed and validated a novel three‐dimensional (3D) printed phantom for evaluating CBCT‐based TF reliability. The phantom has a cylindrical shape (22 cm diameter and 25.5 cm height) with five inner inserts designed to hold custom‐printed rods (1 cm diameter and 10–20 cm height) of various materials, infill shapes, and densities. TF reproducibility was evaluated across and within three LINACs from a single vendor using sets of three consecutive CBCT taken with the head, thorax, and pelvis clinical imaging protocols. PyRadiomics was used to extract a standard set of TFs from regions of interest centered on each rod. Two‐way mixed effects absolute agreement intra‐class correlation coefficient (ICC) was used to evaluate TF reproducibility, with features showing ICC values above 0.9 considered robust if their Bonferroni‐corrected p‐value was below 0.05. Results A total of 63, 87, and 83 features exhibited test–retest reliability for the head, thorax, and pelvis imaging protocols respectively. When assessing stability between discreet imaging sessions on the same LINAC, these numbers were reduced to 5, 63, and 70 features, respectively. The thorax and pelvis protocols maintained a rich candidate feature space in inter‐LINAC analysis with 61 and 65 features, respectively, exceeding the ICC criteria. Crucially, no features were deemed reproducible when compared between protocols. Conclusions We have developed a 3D phantom for consistent evaluation of TF stability and reproducibility. For LINACs from a single vendor, our study found a substantial number of features available for robust radiomics modeling from CBCT imaging. However, some features showed variations across LINACs. Studies involving CBCT‐based radiomics must preselect features prior to their use in clinical‐based models.

In Vivo Repeatability and Multiscanner Reproducibility of MRI Radiomics Features in Patients With Monoclonal Plasma Cell Disorders

“…Few studies prospectively performed test-retest experiments to investigate repeatability only, but not reproducibility across multiple centers 10–14 . When multiscanner reproducibility was assessed so far, studies relied on phantom measurements only 15–21 . Phantoms offer precious advantages as known geometry and composition, and allow for limitless measurements under a wide variation of image acquisition parameters.…”

Identification of reproducible radiomic features from on‐board volumetric images: A multi‐institutional phantom study

“…For the evaluation of inter-machine analysis, previous studies reported that there are limitations to performing multi-institutional radiomics analysis because inter-machine differences reduce feature reproducibility. 16,17 Our results also showed that feature reproducibility was lower for different types of machines (Halcyon vs. TrueBeam) than for the same types of devices (TrueBeam vs. TrueBeam), even when using the same image reconstruction algorithm (Figure 4). One possible reason for this is that the anti-scatter grid is different: 15:1 for Halcyon and 10:1 for TrueBeam.…”

“…Several researchers have previously designed radiomics phantoms and investigated the repeatability and reproducibility of kV-CT-based radiomic features to establish a basis for analysis. [27][28][29][30] Similarly, although radiomics phantom studies also identified that differences in treatment machines decreased the reproducibility of radiomic features on kV-CBCT, 16,17 there were some concerns regarding their phantom design and lack of external validations. To address these issues, the proposed radiomics phantom simulates a patient's body shape and is sufficiently long in the SI direction.…”

“…16 Spuhler et al designed a three-dimensional (3D)-printed phantom and found that different kV-CBCT scanning devices resulted in more vulnerable radiomic features than intra-and inter-sessions. 17 However, these previous phantom studies did not simulate the patient's body and were only examined at a single center. Because the image quality of kV-CBCT, MV-CBCT, and MV-CT depends on the structural materials surrounding the target, such as the lung and bone in image reconstruction, phantom experiments should be conducted with an anthropomorphic radiomics phantom.…”

“…However, no previous studies have identified the impact of imaging protocols on the reproducibility of radiomic features from on-board volumetric images using an appropriate radiomic phantom structure. Furthermore, it remains unclear how the findings of previous studies [13][14][15][16][17] would change the reproducibility of radiomic features when externally validated using various types of treatment machines at other institutions.…”