Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment

Raunig, David; McShane, Lisa M.; Pennello, Gene; Gatsonis, Constantine; Carson, Paul L.; Voyvodic, James T.; Wahl, Richard L.; Kurland, Brenda F.; Schwarz, Adam J.; Gönen, Mithat; Zahlmann, Gudrun; Kondratovich, Marina; O'Donnell, Kevin; Petrick, Nicholas; Cole, Patricia E.; Garra, Brian S.; Sullivan, Daniel C.

doi:10.1177/0962280214537344

Cited by 269 publications

(358 citation statements)

References 40 publications

(69 reference statements)

Supporting

Mentioning

353

Contrasting

Unclassified

Order By: Relevance

“…Under certain assumptions presented in Li et al (6), the RC was estimated as 1.96 2 2 w σ , where 2 w σ is the estimate of the within-subject variance (6). The 95% CI of RC was calculated based on chi-square statistics as defined in Raunig et al (34). Values for RC were converted to percentage (RC%) for easier interpretation.…”

Section: Discussionmentioning

confidence: 99%

“…Prior to statistical analysis, all volume estimates were log-transformed (natural log) to make the data better suited for subsequent analyses such as ANOVA which assumes homoscedasticity (26,34). Analyses included the extraction of metrics to describe bias and repeatability, as recommended by the QIBA metrology group (27,34).…”

Section: Discussionmentioning

confidence: 99%

“…Analyses included the extraction of metrics to describe bias and repeatability, as recommended by the QIBA metrology group (27,34). Linearity analysis was not performed due to inclusion of only two nodules sizes (5 and 10 mm).…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Quantitative assessment of nonsolid pulmonary nodule volume with computed tomography in a phantom study

Gavrielides

Berman

Supanich

et al. 2017

Quant. Imaging Med. Surg.

Self Cite

View full text Add to dashboard Cite

Background: To assess the volumetric measurement of small (≤1 cm) nonsolid nodules with computed tomography (CT), focusing on the interaction of state of the art iterative reconstruction (IR) methods and dose with nodule densities, sizes, and shapes. Results: Density had the most important effect on measurement error followed by the interaction of density with nodule size. The nonsolid −630 HU nodules had high accuracy and precision at levels comparable to solid (−10 HU) nonsolid, regardless of reconstruction method and with CTDI vol as low as 0.6 mGy. PB was <5% and <11% for the 10-and 5-mm in nominal diameter −630 HU nodules respectively, and RC was <5% and <12% for the same nodules. For nonsolid −800 HU nodules, PB increased to <11% and <30% for the 10-and 5-mm nodules respectively, whereas RC increased slightly overall but varied widely across dose and reconstruction algorithms for the 5-mm nodules. Model-based IR improved measurement accuracy for the 5-mm, low-density (−800, −630 HU) nodules. For other nodules the effect of reconstruction method was small. Dose did not affect volumetric accuracy and only affected slightly the precision of 5-mm nonsolid nodules.Conclusions: Reasonable values of both accuracy and precision were achieved for volumetric measurements of all 10-mm nonsolid nodules, and for the 5-mm nodules with −630 HU or higher density, when derived from scans acquired with below screening dose levels as low as 0.6 mGy and regardless of reconstruction algorithm.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Quantitative assessment of nonsolid pulmonary nodule volume with computed tomography in a phantom study

Gavrielides

Berman

Supanich

et al. 2017

Quant. Imaging Med. Surg.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The goal of this study was to evaluate the performance of the algorithms in terms of their bias, repeatability, and reproducibility [31][32][33] as well as to obtain insights into the underlying reasons for differences between algorithms on a voxel level.…”

Section: Statistical Analyses and Metricsmentioning

confidence: 99%

A Comparison of Lung Nodule Segmentation Algorithms: Methods and Results from a Multi-institutional Study

et al. 2016

View full text Add to dashboard Cite

Tumor volume estimation, as well as accurate and reproducible borders segmentation in medical images, are important in the diagnosis, staging, and assessment of response to cancer therapy. The goal of this study was to demonstrate the feasibility of a multi-institutional effort to assess the repeatability and reproducibility of nodule borders and volume estimate bias of computerized segmentation algorithms in CT images of lung cancer, and to provide results from such a study. The dataset used for this evaluation consisted of 52 tumors in 41 CT volumes (40 patient datasets and 1 dataset containing scans of 12 phantom nodules of known volume) from five collections available in The Cancer Imaging Archive. Three academic institutions developing lung nodule segmentation algorithms submitted results for three repeat runs for each of the nodules. We compared the performance of lung nodule segmentation algorithms by assessing several measurements of spatial overlap and volume measurement. Nodule sizes varied from 29 μl to 66 ml and demonstrated a diversity of shapes. Agreement in spatial overlap of segmentations was significantly higher for multiple runs of the same algorithm than between segmentations generated by different algorithms (p < 0.05) and was significantly higher on the phantom dataset compared to the other datasets (p < 0.05). Algorithms differed significantly in the bias of the measured volumes of the phantom nodules (p < 0.05) underscoring the need for assessing performance on clinical data in addition to phantoms. Algorithms that most accurately estimated nodule volumes were not the most repeatable, emphasizing the need to evaluate both their accuracy and precision. There were considerable differences between algorithms, especially in a subset of heterogeneous nodules, underscoring the recommendation that the same software be used at all time points in longitudinal studies.

show abstract

“…ADC values was calculated from a fixed circle area (8-mm diameter) placed on the thyroid gland of human subject. The within-subject coefficient of variation (wCV) was calculated to assess the repeatability of the ADC measurement by using the Quantitative Imaging Biomarker Alliance (Radiological Society of North America) proposed statistical metrics (29,30): wCV = r w /l, where l is the overall mean of ADC measurement and, r w is the standard deviation of ADC measurements within subject.…”

Section: Mri Studymentioning

confidence: 99%

Using Diffusion-Weighted MRI to Predict Aggressive Histological Features in Papillary Thyroid Carcinoma: A Novel Tool for Pre-Operative Risk Stratification in Thyroid Cancer

Moreira

Hatzoglou

et al. 2015

Thyroid

View full text Add to dashboard Cite

Background: Initial management recommendations of papillary thyroid carcinoma (PTC) are very dependent on preoperative studies designed to evaluate the presence of PTC with aggressive features. The purpose of this study was to evaluate whether diffusion-weighted magnetic resonance imaging (DW-MRI) before surgery can be used as a tool to stratify tumor aggressiveness in patients with PTC. Methods: In this prospective study, 28 patients with PTC underwent DW-MRI studies on a three Tesla MR scanner prior to thyroidectomy. Due to image quality, 21 patients were finally suitable for further analysis. Apparent diffusion coefficients (ADCs) of normal thyroid tissues and PTCs for 21 patients were calculated. Tumor aggressiveness was defined by surgical histopathology. The Mann-Whitney U test was used to compare the difference in ADCs among groups of normal thyroid tissues and PTCs with and without features of tumor aggressiveness. Receiver operating characteristic (ROC) analysis was performed to assess the discriminative specificity, sensitivity, and accuracy of and determine the cutoff value for the ADC in stratifying PTCs with tumor aggressiveness. Results: There was no significant difference in ADC values between normal thyroid tissues and PTCs. However, ADC values of PTCs with extrathyroidal extension (ETE; 1.53 -0.25 · 10 -3 mm 2 /s) were significantly lower than corresponding values from PTCs without ETE (2.37 -0.67 · 10 -3 mm 2 /s; p < 0.005). ADC values identified 3 papillary carcinoma patients with extrathyroidal extension that would have otherwise been candidates for observation based on ultrasound evaluations. The cutoff value of ADC to discriminate PTCs with and without ETE was determined at 1.85 · 10 -3 mm 2 /s with a sensitivity of 85%, specificity of 85%, and ROC curve area of 0.85. Conclusion: ADC value derived from DW-MRI before surgery has the potential to stratify ETE in patients with PTCs.

show abstract

Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment

Cited by 269 publications

References 40 publications

Quantitative assessment of nonsolid pulmonary nodule volume with computed tomography in a phantom study

Quantitative assessment of nonsolid pulmonary nodule volume with computed tomography in a phantom study

A Comparison of Lung Nodule Segmentation Algorithms: Methods and Results from a Multi-institutional Study

Using Diffusion-Weighted MRI to Predict Aggressive Histological Features in Papillary Thyroid Carcinoma: A Novel Tool for Pre-Operative Risk Stratification in Thyroid Cancer

Contact Info

Product

Resources

About