Is bias in the eye of the beholder? A vignette study to assess recognition of cognitive biases in clinical case workups

Abstract

BackgroundMany authors have implicated cognitive biases as a primary cause of diagnostic error. If this is so, then physicians already familiar with common cognitive biases should consistently identify biases present in a clinical workup. The aim of this paper is to determine whether physicians agree on the presence or absence of particular biases in a clinical case workup and how case outcome knowledge affects bias identification.MethodsWe conducted a web survey of 37 physicians. Each participant read eight cases and listed which biases were present from a list provided. In half the cases the outcome implied a correct diagnosis; in the other half, it implied an incorrect diagnosis. We compared the number of biases identified when the outcome implied a correct or incorrect primary diagnosis. Additionally, the agreement among participants about presence or absence of specific biases was assessed.ResultsWhen the case outcome implied a correct diagnosis, an average of 1.75 cognitive biases were reported; when incorrect, 3.45 biases (F=71.3, p<0.00001). Individual biases were reported from 73% to 125% more often when an incorrect diagnosis was implied. There was no agreement on presence or absence of individual biases, with κ ranging from 0.000 to 0.044.InterpretationIndividual physicians are unable to agree on the presence or absence of individual cognitive biases. Their judgements are heavily influenced by hindsight bias; when the outcome implies a diagnostic error, twice as many biases are identified. The results present challenges for current error reduction strategies based on identification of cognitive biases.

Premature closure? Not so fast

“…A study in this issue point out how even this may not be true 14. In a web-based study of 37 physicians, Zwaan et al demonstrated that when clinicians reviewed case vignettes, they were more likely to ‘see’ cognitive errors when they learned that the working diagnosis was incorrect than when it was correct (and regardless, clinicians did not agree on which cognitive errors were present).…”

Why do we love to hate ourselves?

“…Dhaliwal's comment1 on Zwaan et al 2 nicely refutes what has been called ‘the hypothesis of special cause’3—the notion that when things turn out wrong, the cognitive processes leading to that outcome must have been fundamentally different (ie, error-prone) from when they turn out right. Dhaliwal's argument recapitulates thinking that is over 100 years old; one of the early contributors to psychology, Ernst Mach,4 wrote (in 1905): ‘Knowledge and error flow from the same mental source; only success can tell one from the other’.…”

Variations in GPs' decisions to investigate suspected lung cancer: a factorial experiment using multimedia vignettes

“…Direct observation of real physician–patient encounters offers no opportunity to control patients' clinical and sociodemographic characteristics, and so requires observation of very large numbers of consultations to obtain the necessary numbers in specific risk or demographic categories. The use of fictional patient profiles (vignettes) can provide a valid and efficient approach to examining clinician behaviour,11 and studies have already produced useful insights into sources of error in clinicians' decision-making processes, due to both patient factors (eg, symptom characteristics)12 and physician factors (eg, cognitive biases) 12 13. As Blumenthal-Barby and others recognise, however, there are limits to the applicability of written vignettes and other vignette designs that do not simulate key features of real consultations 14.…”