Biases in the Evaluation of Psychiatric Clinical Evidence

Abstract

The evolution of medical research has vaulted randomized clinical trials to the status of current gold standard of clinical evidence. In parallel, the evolution of the science of decision-making has revealed human beings' universal tendency to make biased judgments and systematic errors in their evaluation of information and choices. As a result of numerous psychological biases, randomized clinical trials are more prone to error, misinterpretation, and faulty judgment than is often acknowledged. Interdisciplinary fields of experimental psychology, economics, and social science are drawn upon to examine psychological biases in the interpretation of clinical evidence. A number of these are postulated to be important, both for the investigators generating clinical evidence and for the clinical observers interpreting clinical trials. This study focuses on the field of psychiatry and on the potentially significant implications of evidence biases for psychiatric practice and clinical understanding.

Evaluating the evidence for evidence‐based medicine: are randomized clinical trials less flawed than other forms of peer‐reviewed medical research?

“…Prospective studies appear to have as many methodological limitations as nonprospective studies (Table 3), and RCTs have as many limitations as non‐RCTs (Table 4). Although it has been established that non‐RCT research can be methodologically flawed (1–7), we report that the published results from RCTs are as often flawed as other types of medical research (Table 4).…”

“…Rather, we show that the RCT approach is often compromised by poor implementation or questionable interpretation. Although RCTs in principle offer scientific validity and high confidence in the research design, the implementation of RCT methods and the interpretation of the results can be flawed by poor trial design, observer bias, incentive bias, or simple misinterpretation (7).…”

“…Nevertheless, it has been recognized that reporting of clinical trials can be flawed (5). Although RCTs remain the gold standard to detect differences between active and control treatments (6), problems can arise through poor trial design, observer bias, incentive bias, or misinterpretation (7). Our intention here is to characterize the methodological limitations of RCTs and to compare RCTs to other types of published medical research, hereinafter referred to as non‐RCTs.…”

The Evidence-based Medicine Paradigm: Where are We 20 Years Later? Part 1

“…11,44 The CONSORT flow diagram for the two group parallel RCT is shown in Figure 1, a type that now accounts for half of published trials, 44 an accompanying checklist (available on the website) helps readers and reviewers assess quality. 11 Randomized controlled trials are "human constructs" and therefore "fallible;" hence, deficiencies in design, including errors because of cognitive biases, can occur at any phase of a trial, 30,45 the "poorly constructed trials" Sir Austin warned us about. 34 Despite the CONSORT recommendations, these errors can be missed by editors and reviewers of even "leading high impact journals," 30 blind-siding the trusting clinician.…”

Translating evidence into mental health nursing practice: User‐centred design

“…Clinical practice is influenced by a wide range of factors that may bias clinicians' judgements of which treatments are most effective. Potential sources of bias include the following: the salience of clinicians' own training in and commitment to specific theoretical models and treatment methods; the tendency to over‐generalize from recent clinical cases or idiosyncratic events; pressure or desire to conform to local communities of practice; and a host of general expectancy effects and confirmation biases that serve to justify and maintain previous decisions and courses of action (Makhinson, ). A potential way forward is to implement evidence‐based interventions in a way that recognizes the unique nature of the particular clinical environment in which they will be implemented.…”