A pilot study in ophthalmology of inter-rater reliability in classifying diagnostic errors: an underinvestigated area of medical error

Abstract

Background: Misdiagnosis is the least studied form of medical error. Before effective strategies to reduce misdiagnosis can be developed, there needs to be a better understanding of the factors that lead to these errors. Aim: To evaluate the applicability and reliability of three classification systems for misdiagnosis. Design: Retrospective independent analysis of five cases by clinical experts. Participants: Three ophthalmologists trained in ocular oncology who devote at least 75% of their practice to ocular oncology. Main outcome measures: Percentage agreement in determining cause of misdiagnosis. Results: Participants agreed a misdiagnosis occurred in all cases and the error was graded as serious 14 of 15 times (93%). Inter-rater agreement for root cause varied among the three classification systems from 47% to zero. Conclusions: Although there was excellent agreement among clinical experts of what constitutes serious misdiagnosis under idealized conditions, there is not a reliable method for categorizing the primary or root cause for these errors. The origins of misdiagnosis are complex, often multifactorial, and more difficult to categorize than other types of medical error. Misdiagnosis is a professional and public healthcare challenge that will require novel strategies to enable it to be successfully studied.

Are accreditation surveys reliable?

“…Within healthcare the ability to make reliable judgements is indeed an essential and critical skill. In health fields as diverse as anesthesiology (Schubert et al , 1999), oral medical examination (Kearney et al , 2002), nursing home assessment (Mov et al , 2003), aphasiology (Hula et al , 2003), ophthalmology (Margo, 2003), gait (Maynard et al , 2003), primary care (Thom et al , 2004), stroke (Hand et al , 2006), medical reviews (Hofer et al , 2000) and depression assessment (Persons and Bertagnolli, 1999) health professionals all face a similar challenge: ensuring judgement consistency, individually and collectively. Safety and quality are founded upon health professionals' ability to make judgements, assessments and interpretations.…”

Diagnostic field reliability in forensic mental health evaluations.

“…Diagnostic field reliability in medicine and mental health is generally mediocre, despite continuing advances in both fields. In 2003, approximately 74,000 individuals in the United States died as a result of medical diagnostic disagreement, leading to misdiagnosis and inadequate treatment (Margo, 2003). In mental health settings, diagnostic agreement rates have been shown to range from approximately 50% in an adult sample (Beck, Ward, Mendelson, Mock, & Erbaugh, 1962) to 71.4% in a sample of adolescents in a psychiatric hospital (Hickin, Slate, & Saarnio, 1996).…”

Clinical Decision Making

“…For physicians to believe that they are not vulnerable to bias is blind optimism at best and sheer arrogance at worst. Within the last decade, however, several disciplines have acknowledged the importance of cognitive biases and their impact on clinical reasoning: Anesthesia [ 33 ], Dermatology [ 34 ], Emergency Medicine [ 35 ], Medicine [ 26 , 36 ], Neurology [ 37 ], Obstetrics [ 38 ], Ophthalmology [ 39 ], Pathology [ 40 , 41 ], Pediatrics [ 42 ], Psychiatry [ 43 ], Radiology [ 44 ], Surgery [ 45 ], as well as specialty environments such as Intensive Care Units [ 46 ], and also Dentistry [ 47 ].…”