Is Confidence of Mammographic Assessment a Good Predictor of Accuracy?

Abstract

OBJECTIVE Interpretive accuracy varies among radiologists, especially in mammography. This study examines the relationship between radiologists’ confidence in their assessments and their accuracy in interpreting mammograms. MATERIALS AND METHODS In this study, 119 community radiologists interpreted 109 expert-defined screening mammography examinations in test sets and rated their confidence in their assessment for each case. They also provided a global assessment of their ability to interpret mammograms. Positive predictive value (PPV) and negative predictive value (NPV) were modeled as functions of self-rated confidence on each examination using log-linear regression estimated with generalized estimating equations. Reference measures were cancer status and expert-defined need for recall. Effect modification by weekly mammography volume was examined. RESULTS Radiologists who self-reported higher global interpretive ability tended to interpret more mammograms per week (p = 0.08), were more likely to specialize (p = 0.02) and to have completed a fellowship in breast or women’s imaging (p = 0.05), and had a higher PPV for cancer detection (p = 0.01). Examinations for which low-volume radiologists were “very confident” had a PPV of 2.93 times (95% CI, 2.01–4.27) higher than examinations they rated with neutral confidence. Trends of increasing NPVs with increasing confidence were significant for low-volume radiologists relative to noncancers (p = 0.01) and expert nonrecalls (p < 0.001). A trend of significantly increasing NPVs existed for high-volume radiologists relative to expert nonrecall (p = 0.02) but not relative to noncancer status (p = 0.32). CONCLUSION Confidence in mammography assessments was associated with better accuracy, especially for low-volume readers. Asking for a second opinion when confidence in an assessment is low may increase accuracy.

Decision‐making in the referral process of sonographers in primary care screening centers

“…Our results are comparable to studies on uncertainty in the assessment of mammography images . A low annual volume of mammography assessments leads to a higher frequency of low confidence score and lower negative predictive values in a group of radiologists.…”

Collective Intelligence Meets Medical Decision-Making: The Collective Outperforms the Best Radiologist

“…As this dataset contains the independent assessments by multiple radiologists of the same mammogram (mean number of independent readings per mammogram = 92), and the true status of each mammogram ( S1 Text ), it allows us to investigate the performance of the above CI-rules. We stress that, while the above dataset has recently been used to investigate the performance of individual radiologists [ 29 – 31 ], up to now, its potential to investigate CI in mammography screening has not yet been harnessed.…”

Accuracy of self-monitoring during learning of radiograph interpretation

“…can nonetheless demonstrate significant selfmonitoring ability ('Will I get THIS question right?'). 9 Self-monitoring has also been shown to correlate with accuracy in radiograph interpretation 10,11 (Table 1).…”

“…Eva and Regehr showed that psychology students who have relatively poor self‐assessment ability for a certain knowledge domain (‘Can I predict the number of questions I will get right on this examination?’) can nonetheless demonstrate significant self‐monitoring ability (‘Will I get THIS question right?’) . Self‐monitoring has also been shown to correlate with accuracy in radiograph interpretation (Table ).…”