Bias in Neuroradiology Peer Review: Impact of a “Ding” on “Dinging” Others

Abstract

BACKGROUND AND PURPOSE: The validity of radiology peer review requires an unbiased assessment of studies in an environment that values the process. We assessed radiologists' behavior reviewing colleagues' reports. We hypothesized that when a radiologist receives a discrepant peer review, he is more likely to submit a discrepant review about another radiologist. MATERIALS AND METHODS: We analyzed the anonymous peer review submissions of 13 neuroradiologists in semimonthly blocks of time from 2016 to 2018. We defined a discrepant review as any one of the following: 1) detection miss, clinically significant; 2) detection miss, clinically not significant; 3) interpretation miss, clinically significant; or 4) interpretation miss, clinically not significant. We used randomeffects Poisson regression analysis to determine whether a neuroradiologist was more likely to submit a discrepant report during the semimonthly block in which he or she received one versus the semimonthly block thereafter. RESULTS: Four hundred sixty-eight discrepant peer review reports were submitted; 161 were submitted in the same semimonthly block of receipt of a discrepant report and 325 were not. Receiving a discrepant report had a positive effect on submitting discrepant reports: an expected relative increase of 14% (95% CI, 8%-21%). Notably, receiving a clinically not significant discrepant report (coefficient ϭ 0.13; 95% CI, 0.05-0.22) significantly and positively correlated with submitting a discrepant report within the same time block, but this was not true of clinically significant reports. CONCLUSIONS: The receipt of a clinically not significant discrepant report leads to a greater likelihood of submitting a discrepant report. The motivation for such an increase should be explored for potential bias.

CAR Peer Learning Guide

“…7,10,11 One study found that radiologists were more likely to submit a discrepant peer review report within a semi-monthly block of time after receiving one, suggesting another source of bias. 12 An important drawback to peer review of randomly selected cases is that they rarely reveal clinically significant discrepancies, and therefore have low yield for learning. In a study that reviewed over 42 000 reports over 42 months, the potentially meaningful discrepancy rate was only 0.5%, and usage of the peer review system declined by 56% between the first and last quarters of the study period.…”

“…7,10,11 One study found that radiologists were more likely to submit a discrepant peer review report within a semi-monthly block of time after receiving one, suggesting another source of bias. 12…”