Screening Mammography: Test Set Data Can Reasonably Describe Actual Clinical Reporting

Soh, BaoLin Pauline; Lee, Warwick; McEntee, Mark F.; Kench, Peter; Reed, Warren; Heard, Rob; Chakraborty, Dev P.; Brennan, Patrick C.

doi:10.1148/radiol.13122399

Cited by 42 publications

(28 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is impractical to run an observer study that exactly matches screening, but there is evidence that the performance of experienced mammography readers in observer studies correlates with their performance in screening (24). Two retrospective screening studies (9, 10) calculated the cancer detection rate (CDR) for two detector types: DR (equivalent to combination of a-Se and CsI arms) and CR PIP.…”

Section: Discussionmentioning

confidence: 99%

Breast cancer detection rates using four different types of mammography detectors

et al. 2015

Self Cite

View full text Add to dashboard Cite

Objectives To compare the performance of different types of detectors in breast cancer detection. Methods A mammography image set containing subtle malignant non-calcification lesions, biopsy-proven benign lesions, simulated malignant calcification clusters and normals was acquired using amorphous-selenium (a-Se) detectors. The images were adapted to simulate four types of detectors at the same radiation dose: digital radiography (DR) detectors with a-Se and caesium iodide (CsI) convertors, and computed radiography (CR) detectors with a powder phosphor (PIP) and a needle phosphor (NIP). Seven observers marked suspicious and benign lesions. Analysis was undertaken using jackknife alternative free-response receiver operating characteristics weighted figure of merit (FoM). The cancer detection fraction (CDF) was estimated for a representative image set from screening. Results No significant differences in the FoMs between the DR detectors were measured. For calcification clusters and non-calcification lesions, both CR detectors’ FoMs were significantly lower than for DR detectors. The calcification cluster’s FoM for CR NIP was significantly better than for CR PIP. The estimated CDFs with CR PIP and CR NIP detectors were up to 15% and 22% lower respectively than for DR detectors. Conclusion Cancer detection is affected by detector type and the use of CR in mammography should be reconsidered.

show abstract

Section: Discussionmentioning

confidence: 99%

Breast cancer detection rates using four different types of mammography detectors

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a study of nine radiologists who interpreted 276-300 film-screen mammograms, Gur et al 19 found that screening mammography performance was better and less variable in clinical practice than on the same examinations in a test set, but they did not assess whether test set performance reflected relative performance in clinical practice. In contrast, Soh et al 20 compared the performance of ten radiologists interpreting 200 screening mammograms (up to 20 with cancer) both in clinical practice and on a test set, and they found good agreement if comparison images were available. Comparing performance on the same exams interpreted by different radiologists in different settings, we found test set sensitivity was slightly lower than, but not significantly different from, clinical sensitivity, but that that test set specificity was higher than clinical specificity, reaching statistical significance for two test sets.…”

Section: Discussionmentioning

confidence: 99%

Correlation Between Screening Mammography Interpretive Performance on a Test Set and Performance in Clinical Practice

et al. 2017

View full text Add to dashboard Cite

Rationale and Objectives Evidence is inconsistent about whether radiologists' interpretive performance on a screening mammography test set reflects their performance in clinical practice. This study aimed to estimate the correlation between test-set and clinical performance and determine if the correlation is influenced by cancer prevalence or lesion difficulty in the test set. Materials and Methods This institutional review board–approved study randomized 83 radiologists from six Breast Cancer Surveillance Consortium registries to assess one of four test sets of 109 screening mammograms each; 48 radiologists completed a fifth test set of 110 mammograms 2 years later. Test sets differed in number of cancer cases and difficulty of lesion detection. Test-set sensitivity and specificity were estimated using woman-level and breast-level recall with cancer status and expert opinion as gold standards. Clinical performance was estimated using women-level recall with cancer status as the gold standard. Spearman rank correlations between test-set and clinical performance with 95% confidence intervals (CI) were estimated. Results For test sets with fewer cancers (N=15) that were more difficult to detect, correlations were weak-to-moderate for sensitivity (woman-level=0.46, 95%CI=0.16, 0.69; breast-level=0.35, 95%CI=0.03, 0.61) and weak for specificity (0.24, 95%CI=0.01, 0.45) relative to expert recall. Correlations for test sets with more cancers (N=30) were close to zero and not statistically significant. Conclusions Correlations between screening performance on a test set and performance in clinical practice are not strong. Test set performance more accurately reflects performance in clinical practice if cancer prevalence is low and lesions are challenging to detect.

show abstract

“…Previous studies have not consistently found a correlation between radiologists’ accuracy on test sets and their accuracy in actual clinical practice, while others have found a correlation. [28-30] Another strength of this study is the large size of the BCSC database, which includes data on more than 2.25 million mammography examinations, facilitating reliable sensitivity assessments among a large number of radiologists at the individual level.…”

Section: Discussionmentioning

confidence: 99%

Radiologists' interpretive skills in screening vs. diagnostic mammography: are they related?

et al. 2016

View full text Add to dashboard Cite

Purpose To determine whether radiologists who perform well in screening also perform well in interpreting diagnostic mammography. Materials & Methods We evaluated the accuracy of 468 radiologists interpreting 2,234,947 screening and 196,164 diagnostic mammograms. Adjusting for site, radiologist, and patient characteristics, we identified radiologists with performance in the highest tertile and compared to those with lower performance. Results A moderate correlation was noted for radiologists’ accuracy when interpreting screening versus their accuracy on diagnostic exams: sensitivity (rspearman=0.51, 95% CI: 0.22, 0.80; P=0.0006), specificity (rspearman=0.40, 95% CI: 0.30, 0.49; P<0.0001). Conclusion Different educational approaches to screening and diagnostic imaging should be considered.

show abstract

Screening Mammography: Test Set Data Can Reasonably Describe Actual Clinical Reporting

Abstract: Reasonable levels of agreement between actual clinical reporting and test set conditions can be achieved, although inflated sensitivity may be evident with test set conditions.

Cited by 42 publications

References 18 publications

Breast cancer detection rates using four different types of mammography detectors

Breast cancer detection rates using four different types of mammography detectors

Correlation Between Screening Mammography Interpretive Performance on a Test Set and Performance in Clinical Practice

Radiologists' interpretive skills in screening vs. diagnostic mammography: are they related?

Contact Info

Product

Resources

About