Statistical analysis of lung nodule volume measurements with CT in a large‐scale phantom study

Abstract

Purpose-To determine inter-related factors that contribute substantially to measurement error of pulmonary nodule measurements with CT by assessing a large-scale dataset of phantom scans and to quantitatively validate the repeatability and reproducibility of a subset containing nodules and CT acquisitions consistent with the Quantitative Imaging Biomarker Alliance (QIBA) metrology recommendations. Methods-The dataset has about 40 000 volume measurements of 48 nodules (5-20 mm, four shapes, three radiodensities) estimated by a matched-filter estimator from CT images involving 72 imaging protocols. Technical assessment was performed under a framework suggested by QIBA, which aimed to minimize the inconsistency of terminologies and techniques used in the literature. Accuracy and precision of lung nodule volume measurements were examined by analyzing the linearity, bias, variance, root mean square error (RMSE), repeatability, reproducibility, and significant and substantial factors that contribute to the measurement error. Statistical methodologies including linear regression, analysis of variance, and restricted maximum likelihood were applied to estimate the aforementioned metrics. The analysis was performed on both the whole dataset and a subset meeting the criteria proposed in the QIBA Profile document.Results-Strong linearity was observed for all data. Size, slice thickness × collimation, and randomness in attachment to vessels or chest wall were the main sources of measurement error. Grouping the data by nodule size and slice thickness × collimation, the standard deviation (3.9%-28%), and RMSE (4.4%-68%) tended to increase with smaller nodule size and larger slice thickness. For 5, 8, 10, and 20 mm nodules with reconstruction slice thickness ≤0.8, 3, 3, and 5 mm, respectively, the measurements were almost unbiased (−3.0% to 3.0%). Repeatability coefficients (RCs) were from 6.2% to 40%. Pitch of 0.9, detail kernel, and smaller slice thicknesses yielded better (smaller) RCs than those from pitch of 1.2, medium kernel, and larger slice thicknesses. Exposure showed no impact on RC. The overall reproducibility coefficient (RDC) was 45%, and reduced to about 20%-30% when the slice thickness and collimation were fixed. For nodules and CT imaging complying with the QIBA Profile (QIBA Profile subset), the measurements were highly repeatable and reproducible in spite of variations in nodule characteristics and imaging protocols. The overall measurement error was small and mostly due to a Electronic mail: qin.li1@fda.hhs.gov. HHS Public AccessAuthor manuscript Med Phys. Author manuscript; available in PMC 2017 November 01. Author Manuscript Author Manuscript Author ManuscriptAuthor Manuscript the randomness in attachment. The bias, standard deviation, and RMSE grouped by nodule size and slice thickness × collimation in the QIBA Profile subset were within ±3%, 4%, and 5%, respectively. RCs are within 11% and the overall RDC is equal to 11%. Conclusions-The authors have performed a comprehensive technical assessme...

Quantitative assessment of nonsolid pulmonary nodule volume with computed tomography in a phantom study

“…The repeatability coefficient (RC) is defined as the least significant difference between two repeated measurements at a two-sided significance of α=0.05. Under certain assumptions presented in Li et al (6), the RC was estimated as 1.96 2 2 w σ , where 2 w σ is the estimate of the within-subject variance (6). The 95% CI of RC was calculated based on chi-square statistics as defined in Raunig et al (34).…”

“…The −800 and −630 HU densities represented the low end to middle of the range of the range of nonsolid nodules in clinical practice whereas −10HU was within the typical density range of solid nodules. Spiculated nodules were included since clinically they were shown to have increased probability of malignancy (15) and in terms of measurement they could be a factor contributing to measurement error (6,28). All nodules were placed within foam inserts to avoid any vessel attachment and provide an approximation to the density of the lung parenchyma.…”

“…Volume has been examined for its potential for earlier determination of change in nodule size compared to a measurement of maximum diameter from a single slice (1) typically used in clinical practice. Numerous studies have been conducted to examine and quantify the interrelated effects of imaging protocols, nodule characteristics, and estimation methods on the measurement error of lung nodule volume, as summarized in two reviews on the subject published in 2009 (2) and 2014 (3) and presented in subsequent publications (4)(5)(6)(7)(8)(9)(10)(11). However, most of the aforementioned studies have focused on the measurement of solid pulmonary nodules, defined as those that completely obscure the lung parenchyma (12).…”

“…We limited our review of related work to phantom studies, which enable the analysis of both accuracy and precision due to the availability of a reference standard regarding nodule size and allow the acquisition of multiple repeated scans without concern to patient exposure. Li et al (6), Doo et al (22), and Siegelman et al (23) examined the volumetry of low-density nodules across different imaging protocols, but the lowest nodule density was limited to −630 HU. Scholten et al (18) and Linning et al (24) did not examine the effect of different reconstruction methods beyond FBP.…”

Detectable change of lung nodule volume with CT in a phantom study with high and low signal to background contrast

“…These effects, and the effect of the operator that uses the measurement tool, need to be understood and accounted for before volume can be fully characterized as a metric for estimating tumor burden in clinical practice. Several studies have been examining the aforementioned effects as summarized in a 2009 review by Gavrielides et al (1) and in subsequent studies (2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18).…”