Comparing the evidential strength for psychotropic drugs: a Bayesian meta-analysis

Abstract

Approval and prescription of psychotropic drugs should be informed by the strength of evidence for efficacy. Using a Bayesian framework, we examined (1) whether psychotropic drugs are supported by substantial evidence (at the time of approval by the Food and Drug Administration), and (2) whether there are systematic differences across drug groups. Data from short-term, placebo-controlled phase II/III clinical trials for 15 antipsychotics, 16 antidepressants for depression, nine antidepressants for anxiety, and 20 drugs for attention deficit hyperactivity disorder (ADHD) were extracted from FDA reviews. Bayesian model-averaged meta-analysis was performed and strength of evidence was quantified (i.e. BFBMA). Strength of evidence and trialling varied between drugs. Median evidential strength was extreme for ADHD medication (BFBMA = 1820.4), moderate for antipsychotics (BFBMA = 365.4), and considerably lower and more frequently classified as weak or moderate for antidepressants for depression (BFBMA = 94.2) and anxiety (BFBMA = 49.8). Varying median effect sizes (ESschizophrenia = 0.45, ESdepression = 0.30, ESanxiety = 0.37, ESADHD = 0.72), sample sizes (Nschizophrenia = 324, Ndepression = 218, Nanxiety = 254, NADHD = 189.5), and numbers of trials (kschizophrenia = 3, kdepression = 5.5, kanxiety = 3, kADHD = 2) might account for differences. Although most drugs were supported by strong evidence at the time of approval, some only had moderate or ambiguous evidence. These results show the need for more systematic quantification and classification of statistical evidence for psychotropic drugs. Evidential strength should be communicated transparently and clearly towards clinical decision makers.

The flaw of averages: Bayes factors as posterior means of the likelihood ratio

“…Nonetheless, in recent years, several researchers have begun to re-analyze results from the clinical literature, or perform simulations, with the goals of demonstrating that BFs: (1) often give divergent conclusions from those of null hypothesis significance testing (NHST) using p-values, and (2) are straightforward to interpret as directly quantifying the relative strength of evidence. [5][6][7][8][9][10][11] For example, in a Bayesian meta-analysis of dozens of FDAapproved psychotropic drugs, some of the corresponding BFs indicated ambiguous evidence for efficacy. 8 The authors argued that BFs could help to "set up a consistent and transparent standard for evaluating strength of evidence of efficacy in the approval process…" In a re-analysis of 58 trials for FDA-approved antidepressants in the treatment of anxiety, the BFs were highly variable, ranging from no support to strong evidence for efficacy.…”

“…[5][6][7][8][9][10][11] For example, in a Bayesian meta-analysis of dozens of FDAapproved psychotropic drugs, some of the corresponding BFs indicated ambiguous evidence for efficacy. 8 The authors argued that BFs could help to "set up a consistent and transparent standard for evaluating strength of evidence of efficacy in the approval process…" In a re-analysis of 58 trials for FDA-approved antidepressants in the treatment of anxiety, the BFs were highly variable, ranging from no support to strong evidence for efficacy. 9 The authors further proposed that BFs can be interpreted directly to compare drug efficacy, and help clinicians to select a specific drug to start treatment: "Suppose we… obtain BF 10 ¼ 10 and BF 10 ¼ 100 for Drug A and Drug B, respectively from two independent trials…We can conclude that the data support the efficacy of Drug B 10 times more than Drug A.…”

“…While BFs have been used widely in several scientific and academic disciplines, they are rarely reported in mainstream clinical research. Nonetheless, in recent years, several researchers have begun to re‐analyze results from the clinical literature, or perform simulations, with the goals of demonstrating that BFs: (1) often give divergent conclusions from those of null hypothesis significance testing (NHST) using p ‐values, and (2) are straightforward to interpret as directly quantifying the relative strength of evidence 5–11 . For example, in a Bayesian meta‐analysis of dozens of FDA‐approved psychotropic drugs, some of the corresponding BFs indicated ambiguous evidence for efficacy 8 .…”