Confirmatory Information Seeking is Robust in Psychologists’ Diagnostic Reasoning

Abstract

Objectives: Across two experiments, we examined three cognitive biases (order effects, context effects, confirmatory bias) in licensed psychologists’ diagnostic reasoning. Hypotheses: Our main prediction was that psychologist-participants would seek confirming vs. disconfirming information after forming an initial diagnostic hypothesis, even given multiple opportunities to seek new information in the same case. We also expected individual differences would affect diagnostic reasoning, such that psychologists with lower (vs. higher) cognitive reflection tendencies and larger (vs. smaller) bias blind spots would be more likely to demonstrate confirmatory bias.Method: Study 1 recruited 149 licensed psychologists (M = 18 years of experience; 44% women; 71% White) and exposed them to one of four randomly-assigned vignettes that varied order effects (one set of symptoms in reversed orders) and context effects (court vs. employer referral). They rank-ordered a list of four possible initial diagnostic hypotheses, and received a piped follow-up choice of which of two pieces of information (confirmatory or disconfirmatory) they wanted to test their initial hypothesis. Study 2 (n =131; M = 21 years of experience; 53% men; 68% White) replicated and extended Study 1, following the same procedure except offering three sequential choice opportunities.Results: Both studies found robust confirmatory information-seeking: 92% sought confirmatory information in Study 1, and confirmation persisted across three opportunities in Study 2 (90%, 84%, 77%), though it lowered with each opportunity (generalized logistic mixed regression model F [2, 378] = 3.85, p = 0.02, ηp2 = 0.02). Conclusions: These findings expand a growing body of research on bias in expert judgment. Specifically, psychologists may engage in robust confirmation bias in the process of forming diagnoses. Although further research is needed on bias and its impact on accuracy, psychologists may need to take steps to reduce confirmatory reasoning processes, such as documenting evidence for and against each decision element.Public Significance Statement: These studies revealed strong evidence that once psychologists form an initial diagnosis, they seek confirming information in an effort to determine if their diagnosis is correct (rather than seeking information to determine if their diagnosis is incorrect). Confirmation bias is a problem because it can lead to misdiagnosis, failure to receive proper treatment, unjust sanctions, and more. Psychologists should take steps to reduce confirmatory bias in their diagnostic reasoning, such as carefully considering and documenting evidence both for and against each element of their decision process.