Evaluating the evidential value of empirically supported psychological treatments (ESTs): A meta-scientific review.

Abstract

Empirically supported treatments (or therapies; ESTs) are the gold standard in therapeutic interventions for psychopathology. Based on a set of methodological and statistical criteria, the APA has assigned particular treatment-diagnosis combinations EST status and has further rated their empirical support as Strong, Modest, and/or Controversial. Emerging concerns about the replicability of research findings in clinical psychology highlight the need to critically examine the evidential value of EST research. We therefore conducted a meta-scientific review of the EST literature, using clinical trials reported in an existing online APA database of ESTs, and a set of novel evidential value metrics (i.e., rates of misreported statistics, statistical power, R-Index, and Bayes Factors). Our analyses indicated that power and replicability estimates were concerningly low across almost all ESTs, and individually, some ESTs scored poorly across multiple metrics, with Strong ESTs failing to continuously outperform their Modest counterparts. Lastly, we found evidence of improvements over time in statistical power within the EST literature, but not for the strength of evidence of EST efficacy. We describe the implications of our findings for practicing psychotherapists and offer recommendations for improving the evidential value of EST research moving forward. General Scientific Summary: This review suggests that although the underlying evidence for a small number of empirically supported therapies is consistently strong across a range of metrics, the evidence is mixed or consistently weak for many, including some classified by Division 12 of the APA as "Strong."

Replication Target Selection in Clinical Psychology: A Bayesian and Qualitative Re-evaluation

“…Published studies for which strength of evidence is low may be in need of replication. In a first attempt to analyze evidence regarding published studies, Sakaluk, Williams, Kilshaw, and Rhyner (2019) conducted a meta-scientific review of selected empirically supported treatments (ESTs). To this end, they used rates of misreporting, estimates of statistical power, R-Index values, and Bayes Factors (BF ).…”

Do We Report the Information that is Necessary to Give Psychology Away? A Scoping Review of the Psychological Intervention Literature 2000-2018

“…Further, we reviewed only peer-reviewed journal articles, from a limited time window (2000-2018) for five categories of psychological interventions, interventions that disproportionately draw from the developmental, educational, social, and personality psychology literature. Our claims should be interpreted with those caveats in mind-it is plausible, and even probable, that they do not generalize to all areas of psychology (though see Sakaluk, Williams, & Kilshaw, 2019) for related concerns in clinical psychology). Moreover, we evaluated implementation readiness through the lens of one implementation science framework-RE-AIM; conclusions may differ if the papers were to be coded with a different implementation science lens -we encourage others to use our open dataset and test this possibility.…”

La Brecha Entre La Investigación Y La Práctica Clínica: Hacia La Integración De La Psicoterapia

“…Es cierto que, en el momento actual, no existe mejor marco de referencia que el método científico para tratar de construir un lenguaje común que permita separar el grano de la paja y establecer ciertos criterios de demarcación. Sin embargo, bajo el halo de la cientificidad también se esconden prácticas de actuación cuestionables que ponen a la psicoterapia en el punto de mira de la crisis de replicabilidad de la ciencia (Leichsenring et al, 2017;Sakaluk et al, 2019;Tackett, Brandes, King y Markon, 2019). Además, existe un desba-LA BRECHA ENTRE INVESTIGACIÓN Y PRÁCTICA lance en el desarrollo de la ciencia de la psicoterapia a favor de los datos empíricos, en detrimento de la construcción conceptual y la plausibilidad de la lógica que sustenta un tratamiento (Berg y Slaattelid, 2017).…”