Effect of CAD on Radiologists' Detection of Lung Nodules on Thoracic CT Scans: Analysis of an Observer Performance Study by Nodule Size

Abstract

Rationale and Objectives-To retrospectively investigate the effect of a computer aided detection (CAD) system on radiologists' performance for detecting small pulmonary nodules in CT examinations, with a panel of expert radiologists serving as the reference standard.Materials and Methods-Institutional review board approval was obtained. Our data set contained 52 CT examinations collected by the Lung Image Database Consortium, and 33 from our institution. All CTs were read by multiple expert thoracic radiologists to identify the reference standard for detection. Six other thoracic radiologists read the CT examinations first without, and then with CAD. Performance was evaluated using free-response receiver operating characteristics (FROC) and the jackknife FROC analysis methods (JAFROC) for nodules above different diameter thresholds.Results-241 nodules, ranging in size from 3.0 to 18.6 mm (mean 5.3 mm) were identified as the reference standard. At diameter thresholds of 3, 4, 5, and 6 mm, the CAD system had a sensitivity of 54%, 64%, 68%, and 76%, respectively, with an average of 5.6 false-positives (FPs) per scan. Without CAD, the average figures-of-merit (FOMs) for the six radiologists, obtained from JAFROC analysis, were 0.661, 0.729, 0.793 and 0.838 for the same nodule diameter thresholds, respectively. With CAD, the corresponding average FOMs improved to 0.705, 0.763, 0.810 and 0.862, respectively. The improvement achieved statistical significance for nodules at the 3 and 4 mm thresholds (p=0.002 and 0.020, respectively), and did not achieve significance at 5 and 6 mm (p=0.18 and 0.13, respectively). At a nodule diameter threshold of 3 mm, the radiologists' average sensitivity and FP rate were 0.56 and 0.67, respectively, without CAD, and 0.67 and 0.78 with CAD.Conclusion-CAD improves thoracic radiologists' performance for detecting pulmonary nodules under 5 mm on CT examinations, which are often overlooked by visual inspection alone.

Computer-Aided Nodule Detection and Volumetry to Reduce Variability Between Radiologists in the Interpretation of Lung Nodules at Low-Dose Screening Computed Tomography

“…8, 26–28 Lung nodule volumetric measurement with CAD facilitates a reduction in interobserver variability in the evaluation of indeterminate nodules in low-dose CT. 12, 13 In the present study, readers changed their decisions on average 19 times in assessing 134 cases (14%) after reviewing CAD results. Decisions were altered more frequently because of CAD nodule size measurement than the detection of new nodules.…”

Automated Detection of Sclerotic Metastases in the Thoracolumbar Spine at CT

“…In addition, preliminary study on the feasibility of CAD of vertebral column metastases at magnetic resonance (MR) imaging and preclinical animal studies of C omputer-aided detection (CAD) techniques and algorithms for radiologic applications are rapidly growing in scope and sophistication (1,2). In computed tomographic (CT) colonography, mammography, and thoracic CT imaging, the potential for CAD to help increase the detection sensitivity for polyps (3), improve the sensitivity for cancer detection and the accuracy for lesion classification in mammography (4,5), and to enhance reader performance in pulmonary nodule detection and assessment of likelihood of malignancy (6,7) has been previously demonstrated.…”

Lung Nodule and Cancer Detection in Computed Tomography Screening

“…Of greatest benefit for facilitating detection of smaller nodules (25, 57, 60), CAD consistently detects nodules that are not seen by radiologists and when used to augment radiologist’s readings, substantially reduces inter-observer variability (59). While the focus of most CAD development has been directed toward solid nodules, systems tuned to the detection of ground glass and part solid nodules are emerging (63–65), and have been shown to improve reader performance for all three classes of lung nodule (64).…”