Assessment of cognitive biases and biostatistics knowledge of medical residents: a multicenter, cross-sectional questionnaire study

Abstract

PurposeThe aim of this study is to determine the perceived familiarity of medical residents with statistical concepts, assess their ability to integrate these concepts in clinical scenarios, and investigate their susceptibility to the gambler’s fallacy and the conjunction fallacy.MethodsA multi-institutional, cross-sectional survey of Greek medical residents was performed. Participants were asked to indicate their familiarity with basic statistical concepts and answer clinically oriented questions designed to assess their biostatistics knowledge and cognitive biases. Univariate, bivariate, and multivariate statistical models were used for the evaluation of data.ResultsOut of 153 respondents (76.5% response rate), only two participants (1.3%) were able to answer all seven biostatistics knowledge questions correctly while 29 residents (19%) gave incorrect answers to all questions. The proportion of correct answers to each biostatistics knowledge question ranged from 15 to 51.6%. Residents with greater self-reported familiarity were more likely to perform better on the respective knowledge question (all p<0.01). Multivariate analysis of the effect of individual resident characteristics on questionnaire performance showed that previous education outside Greece, primarily during medical school, was associated with lower biostatistics knowledge scores (p<0.001). A little more than half of the respondents (54.2%) answered the gambler’s fallacy quiz correctly. Residents with higher performance on the biostatistics knowledge questions were less prone to the gambler’s fallacy (odds ratio 1.38, 95% confidence intervals 1.12–1.70, p=0.003). Only 48 residents (31.4%) did not violate the conjunction rule.ConclusionsA large number of medical residents are unable to correctly interpret crucial statistical concepts that are commonly found in the medical literature. They are also especially prone to the gambler’s fallacy bias, which may undermine clinical judgment and medical decision making. Formalized systematic teaching of biostatistics during residency will be required to de-bias residents and ensure that they are proficient in understanding and communicating statistical information.

Plastic Surgery Residents' Understanding and Attitudes Toward Biostatistics

“…Second, similar studies among other specialties have yielded consistent results with similarly sized samples. 16,[19][20][21] This latter observation suggests that the core deficiency is not necessarily with plastic surgery training, but with postgraduate health professions training in general, regardless of graduate field (medicine, dentistry, pharmacy). In addition, the resident demographics of the respondents suggest a higher proportion of integrated respondents than the population at large and a greater proportion of respondents with advanced degrees, compared to the population at large (22% vs approximately 10%).…”

“…[14][15][16][17][18][19][20][21][22] Even among those with formal training in biostatistics or epidemiology, there was a poor understanding of frequently used statistical tests and a limited ability to correctly interpret the results and identify the associated limitations. With the increasing volume and complexity of statistical data presented in journal articles, it is possible that practitioners with limited knowledge of biostatistics will have increasing difficulty with interpreting the results of current studies.…”

“…22 This is similar to results seen among internal medicine, family practice, oral and maxillofacial surgery, and pharmacy residents. [14][15][16][17][18][19][20][21][22] The purpose of the present study was to assess plastic surgery resident attitudes and knowledge of biostatistics using a national survey. We hypothesized that plastic surgery residents would place a high value and have generally positive attitudes regarding biostatistics, but would only have fair confidence and understanding of core statistical concepts.…”

Cognitive biases associated with medical decisions: a systematic review

“…Only three manuscripts evaluated more than 5 cognitive biases in the same study, in-line with the narrow scope of most studies [39, 50, 52]. One third of studies ( n  = 6) were descriptive, i.e., they provided the frequency of the cognitive bias without outcome data [36, 37, 39, 44, 48, 51]. …”

“…Moreover, across studies we often found a lack (in 30 % of studies) or heterogeneity in the outcome measures, mixed denominators (some studies report their findings based on the number of participants, while others based on case-scenarios) [41, 43, 52], and different scope (e.g. some studies are descriptive, [36, 37, 39, 44, 48, 51] whereas others [7, 30, 35, 42, 43, 47, 50, 52] target diagnostic or therapeutic errors). Third, most studies use hypothetical case-vignettes which may not truly reflect medical decisions in real life.…”