Certain performance values arising from mammographic test set readings correlate well with clinical audit

Abstract

Performance indicators from test set demonstrate significant correlations with specific aspects of clinical performance, although caution needs to be exercised when generalising test set specificity to the clinical situation.

Correlation Between Screening Mammography Interpretive Performance on a Test Set and Performance in Clinical Practice

“…7-9 Similar to Soh et al, 7 we found stronger correlations for sensitivity than for specificity. This is in contrast to Rutter and Taplin, 9 who found moderate correlations for specificity but no evidence of correlation between clinical and test-set sensitivity.…”

“…7-9 In a study of 27 U.S. radiologists who interpreted a test set of 113 film screening mammography examinations (30 with cancer), Rutter and Taplin 9 found moderate correlation between the specificity of screening mammography interpreted in clinical and test settings (0.41; 95% Bayesian credible interval (BCI) 0.16, 0.62), but no evidence of correlation between clinical and test-set sensitivity (-0.18, 95% BCI -0.27, 0.59). In contrast, Soh et al 7 found significant, moderate correlations of 0.30-0.57 between several clinical audit measures and two test set measures (location sensitivity and Jackknifing free response operating characteristic figure-of-merit) of 60 cases (20 with cancer) read by 20 radiologists, but no correlation with test set specificity. Similarly, Scott et al 8 found significant, moderate correlations of 0.29-0.41 between several performance measures on the PERFORMs test set and clinical performance among 39 readers in the UK.…”

Audit feedback on reading performance of screening mammograms: An international comparison

“…16 Statistically significant moderate positive correlations have been demonstrated between reader performance at BREAST test sets and performance demonstrated by clinical audit. 17 In the US, a digital versatile disc (DVD) intervention with enhanced cancer cases that also provided immediate feedback to the radiologist 18 was found in a randomized controlled trial to significantly improve interpretive performance in a post-test set. The effect was not tested in clinical practice.…”

Test Sets and Real-Life Performance: Can One Predict the Other?

“…Offering the test set twice per annum allows for a fair estimate of overall ability and reduces the likelihood of outside variables that may affect a reader's performance on a single test set (2). Similarly, Australia and New Zealand offer a test set program called BREAST (BreastScreen Reader Assessment Strategy), which offers some correlation with real-life performance (3). Test sets can provide immediate feedback on cases with pathologically proven or long-term confirmation of disease.…”