Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging

Soffa, David J; Lewis, Rebecca S.; Sunshine, Jonathan H.; Bhargavan, Mythreyi

doi:10.1016/j.jacr.2003.12.017

Cited by 77 publications

(44 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3,[12][13][14] Soffa et al 14 sampled approximately 7000 cases read by 26 radiologists and uncovered a 3% disagreement rate in general radiology, 3.6% in diagnostic mammography, 5.8% in screening mammography, and 4.1% in sonography, yielding the overall error rate of 3.5%. Robinson et al 13 compared reports for skeletal, chest, and abdominal radiographs completed by 3 radiologists and found a 3%-6% average error rate per observer.…”

mentioning

confidence: 99%

Quality Control in Neuroradiology: Discrepancies in Image Interpretation among Academic Neuroradiologists

Babiarz¹,

Yousem²

2011

AJNR Am J Neuroradiol

View full text Add to dashboard Cite

SUMMARY:Prior studies have found a 3%-6% clinically significant error rate in radiology practice. We set out to assess discrepancy rates between subspecialty-trained university-based neuroradiologists. Over 17 months, university neuroradiologists randomly reviewed 1000 studies and reports of previously read examinations of patients in whom follow-up studies were read. The discrepancies between the original and "second opinion" reports were scored according to a 5-point scale: 1, no change; 2, clinically insignificant detection discrepancy; 3, clinically insignificant interpretation discrepancy; 4, clinically significant detection discrepancy; and 5, clinically significant interpretation discrepancy. Of the 1000 studies, 876 (87.6%) showed agreements with the original report. The neuroradiology division had a 2.0% (20/1000; 95% CI, 1.1%-2.9%) rate of clinically significant discrepancies involving 8 CTs and 12 MR images. Discrepancies were classified as vascular (n ϭ 7), neoplastic (n ϭ 9), congenital (n ϭ 2), and artifacts (n ϭ 2). Individual neuroradiologist's scores ranged from 0% to 7.7% Ϯ 2.3% (n ϭ 18). Both CT and MR imaging studies had a discrepancy rate of 2.0%. Our quality assessment study could serve as initial data before intervention as part of a PQI project.ABBREVIATIONS: ABR ϭ American Board of Radiology; ACGME ϭ Accreditation Council for Graduate Medical Education; CI ϭ confidence interval; PQI ϭ practice quality improvement R adiologic detection and interpretation errors will not be fully eliminated until the advent of "perfect diagnostic tests" and "perfect observers." 1,2 In the meantime, radiologists, similar to other physicians, struggle with assessing physician performance and reporting quality, to improve and deliver the best care possible.3 L. Henry Garland pioneered the work on radiologic errors more than 60 years ago. [4][5][6] He uncovered a 30% rate of missed radiologic findings in a series of radiographs with abnormal findings among expert reviewers. Subsequently Garland's results have been replicated by other researchers. [7][8][9] Most interesting, comparable rates of "mistakes" were discovered in other specialties. 10,11 In deriving the radiologic error rate, Garland used exclusively abnormal studies-that is, he tested radiologists in environments in which disease prevalence reached 100%. Because in the typical clinical setting, there are a substantial number of examinations with normal findings, Garland hypothesized that the expected radiologic error rate in everyday practice is closer to 5%.Subsequent studies confirmed the radiologic error rate in all-comers radiology practice to be in the 3%-6% range. 3,12-14Soffa et al 14 sampled approximately 7000 cases read by 26 radiologists and uncovered a 3% disagreement rate in general radiology, 3.6% in diagnostic mammography, 5.8% in screening mammography, and 4.1% in sonography, yielding the overall error rate of 3.5%. Robinson et al 13 compared reports for skeletal, chest, and abdominal radiographs completed by 3 radiologists and found ...

show abstract

mentioning

confidence: 99%

Quality Control in Neuroradiology: Discrepancies in Image Interpretation among Academic Neuroradiologists

Babiarz¹,

Yousem²

2011

AJNR Am J Neuroradiol

View full text Add to dashboard Cite

show abstract

“…3 Other studies that have used a double-blind method in a much larger number of patients have found lower discrepancy rates than those reported here. 1 In this case, we suspect that the medical director or quality assurance radiologist knowingly working within an audit process may have biased, according to a principle of charity, the selective identification of the most serious discrepancies only. Our study being done in an anonymous, nonblaming context, with no intent to identify deviant individuals differs from many quality assurance studies and may explain why more frequent disagreements could be identified.…”

Section: Discussionmentioning

confidence: 99%

“…[1][2][3] The content of these interventions may not be evident, however. In addition, the manner in which the error, discrepancy, and disagreement should be handled both in theory and in clinical practice is evolving.…”

mentioning

confidence: 99%

“…However, there is no consensus on a standard method or protocol for evaluating errors and discrepancies in imaging reports, and rates published in the literature differ widely. [1][2][3][10][11][12][13][14] Multiple variations in study parameters, including sampling sources, methods, imaging modalities, specialties, categories, interpreter training levels, and degrees of blinding, may have contributed to this wide spectrum. 2,3,9 Recently, CT and MR imaging reports of the head, neck, and spine were re-read by staff neuroradiologists, and a 2% clinically significant discrepancy rate was found, an excellent result compared with the 3%-6% radiologic error rates published in general radiology practices.…”

mentioning

confidence: 99%

“…Performance can be measured; deviations and outliers can be identified, and appropriate measures can then be taken to improve performance. [1][2][3] A different vocabulary is used when discrepancies are examined from a scientific point of view. In the typical absence of a criterion standard of "truth," the uncertainty is a reality that must be admitted and taken into account when using imaging reports for clinical decisions.…”

mentioning

confidence: 99%

See 2 more Smart Citations