Quality Control in Neuroradiology: Discrepancies in Image Interpretation among Academic Neuroradiologists

Abstract

SUMMARY:Prior studies have found a 3%-6% clinically significant error rate in radiology practice. We set out to assess discrepancy rates between subspecialty-trained university-based neuroradiologists. Over 17 months, university neuroradiologists randomly reviewed 1000 studies and reports of previously read examinations of patients in whom follow-up studies were read. The discrepancies between the original and "second opinion" reports were scored according to a 5-point scale: 1, no change; 2, clinically insignificant detection discrepancy; 3, clinically insignificant interpretation discrepancy; 4, clinically significant detection discrepancy; and 5, clinically significant interpretation discrepancy. Of the 1000 studies, 876 (87.6%) showed agreements with the original report. The neuroradiology division had a 2.0% (20/1000; 95% CI, 1.1%-2.9%) rate of clinically significant discrepancies involving 8 CTs and 12 MR images. Discrepancies were classified as vascular (n ϭ 7), neoplastic (n ϭ 9), congenital (n ϭ 2), and artifacts (n ϭ 2). Individual neuroradiologist's scores ranged from 0% to 7.7% Ϯ 2.3% (n ϭ 18). Both CT and MR imaging studies had a discrepancy rate of 2.0%. Our quality assessment study could serve as initial data before intervention as part of a PQI project.ABBREVIATIONS: ABR ϭ American Board of Radiology; ACGME ϭ Accreditation Council for Graduate Medical Education; CI ϭ confidence interval; PQI ϭ practice quality improvement R adiologic detection and interpretation errors will not be fully eliminated until the advent of "perfect diagnostic tests" and "perfect observers." 1,2 In the meantime, radiologists, similar to other physicians, struggle with assessing physician performance and reporting quality, to improve and deliver the best care possible.3 L. Henry Garland pioneered the work on radiologic errors more than 60 years ago. [4][5][6] He uncovered a 30% rate of missed radiologic findings in a series of radiographs with abnormal findings among expert reviewers. Subsequently Garland's results have been replicated by other researchers. [7][8][9] Most interesting, comparable rates of "mistakes" were discovered in other specialties. 10,11 In deriving the radiologic error rate, Garland used exclusively abnormal studies-that is, he tested radiologists in environments in which disease prevalence reached 100%. Because in the typical clinical setting, there are a substantial number of examinations with normal findings, Garland hypothesized that the expected radiologic error rate in everyday practice is closer to 5%.Subsequent studies confirmed the radiologic error rate in all-comers radiology practice to be in the 3%-6% range. 3,12-14Soffa et al 14 sampled approximately 7000 cases read by 26 radiologists and uncovered a 3% disagreement rate in general radiology, 3.6% in diagnostic mammography, 5.8% in screening mammography, and 4.1% in sonography, yielding the overall error rate of 3.5%. Robinson et al 13 compared reports for skeletal, chest, and abdominal radiographs completed by 3 radiologists and found ...

Interobserver Agreement in the Interpretation of Outpatient Head CT Scans in an Academic Neuroradiology Practice

“…[1][2][3] The content of these interventions may not be evident, however. In addition, the manner in which the error, discrepancy, and disagreement should be handled both in theory and in clinical practice is evolving.…”

“…However, there is no consensus on a standard method or protocol for evaluating errors and discrepancies in imaging reports, and rates published in the literature differ widely. [1][2][3][10][11][12][13][14] Multiple variations in study parameters, including sampling sources, methods, imaging modalities, specialties, categories, interpreter training levels, and degrees of blinding, may have contributed to this wide spectrum. 2,3,9 Recently, CT and MR imaging reports of the head, neck, and spine were re-read by staff neuroradiologists, and a 2% clinically significant discrepancy rate was found, an excellent result compared with the 3%-6% radiologic error rates published in general radiology practices.…”

“…2,3,9 Recently, CT and MR imaging reports of the head, neck, and spine were re-read by staff neuroradiologists, and a 2% clinically significant discrepancy rate was found, an excellent result compared with the 3%-6% radiologic error rates published in general radiology practices. 3,15,16 To anyone who has studied reliability or precision of diagnostic imaging tests, such levels of disagreement between interpretations may appear unbelievably low. Peer-review quality assurance "errors and discrepancies" and disagreements in reliability studies of imaging test interpretations may not measure the same things.…”

“…Performance can be measured; deviations and outliers can be identified, and appropriate measures can then be taken to improve performance. [1][2][3] A different vocabulary is used when discrepancies are examined from a scientific point of view. In the typical absence of a criterion standard of "truth," the uncertainty is a reality that must be admitted and taken into account when using imaging reports for clinical decisions.…”

“…[1][2][3][10][11][12][13][14] Multiple variations in study parameters, including sampling sources, methods, imaging modalities, specialties, categories, interpreter training levels, and degrees of blinding, may have contributed to this wide spectrum. 2,3,9 Recently, CT and MR imaging reports of the head, neck, and spine were re-read by staff neuroradiologists, and a 2% clinically significant discrepancy rate was found, an excellent result compared with the 3%-6% radiologic error rates published in general radiology practices. 3,15,16 To anyone who has studied reliability or precision of diagnostic imaging tests, such levels of disagreement between interpretations may appear unbelievably low.…”