Computer-aided detection for CT colonography: incremental benefit of observer training

Abstract

The purpose of this study was to investigate the incremental effect of focused training on observer performance when using computer-assisted detection (CAD) software to interpret CT colonography (CTC). Six radiologists who were relatively inexperienced with CTC interpretation underwent 1 day of focused training before reading 20 patient datasets with the assistance of CAD software (ColonCAR 1.3, Medicsight PLC). Sensitivity, specificity and interpretation times were determined and compared with previous performance when reading the same datasets but without the benefit of focused training, using the binomial exact test and Wilcoxon's signed rank test. Per-polyp sensitivity improved after training by 18% overall (95% confidence interval (CI): 14-24%, p<0.001) and was greatest for polyps of 6-9 mm (26%, 95% CI: 18-34%, p<0.001). Absolute sensitivity was 23% (9-36%), 51% (33-71%) and 74% (44-100%) for polyps of <or=5 mm, 6-9 mm and >or=10 mm, respectively. Specificity fell significantly after focused training (median of 5.5 false positives per 20 datasets (interquartile range (IQR): 4-6) post-training vs median of 2.5 (IQR: 1-5) pre-training, p = 0.03). Interpretation time also increased significantly after training (from a median of 9.3 min (IQR: 9.3-14.5 min) to a median of 17.1 min (IQR: 15.4-19.4 min), p = 0.03). In conclusion, one day of training increases observer polyp sensitivity when using CAD for CTC at the expense of increased reporting time and reduction in specificity.

Accuracy of computed tomographic colonography in a nationwide multicentre trial, and its relation to radiologist expertise

“…This number varied from 1000,25 25–300 26 27 38 or 50 12 28 for the largest studies. Some national guidelines 29 30 recommend that, for certification, the radiologist should analyse 50–75 CTCs and then take a test based on a set of 20 cases, whereas the American Gastroenterological Association recommends training that involves review and interpretation of at least 75 CTCs with endoscopic correlation, and further formal mentoring with at least 25–50 additional cases 3.…”

Effect of Computer-aided Detection for CT Colonography in a Multireader, Multicase Trial

“…Important differences in the study designs when comparing our multireader, multicase study with the study of Petrick et al (10) include the large size of the cohort (100 cases), the large number of readers (19 observers) including nonexperts, the inclusion of nonneoplastic polyps in the cohort, nonexclusion of cases known to also contain polyps smaller than 6 mm in size, and the use of both primary 3D and primary 2D reading methods. When one reads CT colonographic images, the histologic features of a polyp are not known a priori, and, thus, statistical analyses polyps 6 mm or larger or 10 mm or larger in size at CT colonography, because polyps, and specifi cally adenomas, of those sizes have been accepted as the key metrics in the evaluation of effectiveness of CT colonography as a tool to screen for colorectal cancer ( 10,23,32 ). The improved reader sensitivity of 0.055 (11.8%) for patients with both small and large polyps and of greater than 0.08 for small adenomas when using CAD are favorable when compared with results in prior studies.…”

“…There is no single standard metric for comparing results in clinical CAD trials, which is why it is important to present the results by using several different metrics. There are results of remarkably few similar studies about the use of CAD with which to compare our results, partly because most clinical studies about CAD included much smaller cohorts of patients and readers and methods that focused on a specifi c question, such as CAD reading mode, CAD for training ( 12,32 ), or use in different or special viewing methods (3)(4)(5)(6)(7)(8)(9)(10)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)33 ). The largest prior multireader, multicase reader CAD trial in a screening cohort, that of Petrick et al ( 10 ), was conducted by using a subset of 60 patients in which four readers used CAD in a second-reader mode and analyzed only neoplastic polyps (not all polyps * Small polyps are larger than 6 mm but smaller than 10 mm.…”

CAD: How it works, how to use it, performance

“…3 of 7 readers) [12]. However, it has been shown that training reduces the number of false-positive diagnoses by readers [17]. Therefore, there is compelling evidence that the level of reader experience affects their ultimate performance when using CAD.…”