Equivalence and non-inferiority testing in psychotherapy research

Leichsenring, Falk; Abbass, Allan; Driessen, Ellen; Hilsenroth, Mark J.; Luyten, Patrick; Rabung, Sven; Steinert, Christiane

doi:10.1017/s0033291718001289

Cited by 12 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is well known in the methodological literature, where the appropriate test would be either a non-inferiority or equivalence test (Greene, Morland, Durkalski, & Frueh, 2008;Piaggio et al, 2006; Wellek, 2010). Non-inferiority and equivalence test have increasingly been used by psychotherapy researchers (e.g., Leichsenring et al, 2018;Steinert, Munder, Rabung, Hoyer, & Leichsenring, 2017), e.g., when comparing PDT vs. CBT (Driessen et al, 2013), or when comparing an internet-delivered treatment versus a face-to-face treatment (Lappalainen et al, 2014). but clinically meaningless effect exists the sample size would need to be even larger (c.f., Julious, 2004).…”

Section: Non-inferiority and Equivalence Studiesmentioning

confidence: 99%

“…(p. 1393), and went so far as to recommend that Δ should be "90% of the expected effects of the first-line treatments (e.g., a threshold SMD of ±0.05, if the uncontrolled effect size is expected as SMD = 0.50)." Clearly, a Δ = 0.05 will protect against degradation; however, as noted by Leichsenring et al (2018) this would require 6,281 particpats per arm to reach 80% power.…”

Section: Non-inferiority and Equivalence Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Internet-delivered cognitive-behavioral therapy for significant others of treatment-refusing problem gamblers: A randomized wait-list controlled trial.

Magnusson¹,

Nilsson²,

Andersson³

et al. 2019

Journal of Consulting and Clinical Psychology

View full text Add to dashboard Cite

Over the last couple of decades evidence-based psychotherapies have flourished, and there are now therapies that are well-established for a wide range of problems. At the same time the mental-health burden is still enormous, and challenges to the dissemination of treatments are substantial. Despite the considerable gains in knowledge that have been made, many issues remain unsolved, and there are many reasons to be skeptical of the current quality of the evidence.

show abstract

Section: Non-inferiority and Equivalence Studiesmentioning

confidence: 99%

Section: Non-inferiority and Equivalence Studiesmentioning

confidence: 99%

Internet-delivered cognitive-behavioral therapy for significant others of treatment-refusing problem gamblers: A randomized wait-list controlled trial.

Magnusson¹,

Nilsson²,

Andersson³

et al. 2019

Journal of Consulting and Clinical Psychology

View full text Add to dashboard Cite

show abstract

“…For this approach, the definition of equivalence or non-inferiority margins (NIM) is crucial, as we had noted earlier (Rief and Hofmann, 2018). Leichsenring et al (2018) opposed our arguments, based on other examples and their own trials advocating for psychodynamic treatments.First, we want to thank Leichsenring and colleagues for their thorough report and interest on our paper. The authors present an impressive variety of NIMs that have been used in prior publications, confirming our argument that there is no clear consensus for defining NIMs.…”

mentioning

confidence: 81%

mentioning

confidence: 81%

The limitations of equivalence and non-inferiority trials

Rief

Hofmann

2018

Psychol. Med.

View full text Add to dashboard Cite

Equivalence and non-inferiority trials are becoming more and more popular. Typically, they compare the effects of a treatment of interest with the current gold-standard treatment as the comparator. However, for this approach, the definition of equivalence or non-inferiority margins (NIM) is crucial, and no clear rules for their definition exist. We criticized the practice of these trials of being over-inflationary in favor of (erroneous) equivalence, and we outlined our critique with some study examples comparing psychodynamic treatments with current firstline treatments for mental disorders. Here we answer to a commentary of Leichsenring et al. to our paper. Although focusing on our commentary, these authors are less arguing against our conclusions, but they address issues of study conduct, and lack of appreciation of our examples. However, the crucial question is: What is the risk of erroneous equivalence conclusions that we want to accept as responsible clinicians and scientists? We conclude that the scientific community has to define better and clearer criteria for NIMs. We do not believe that it is ethically justifiable to recommend a treatment that is 10 or 20% less effective than the current gold standard interventions.Equivalence and non-inferiority trials typically compare the effects of a treatment of interest with the current gold-standard treatment as the comparator. For this approach, the definition of equivalence or non-inferiority margins (NIM) is crucial, as we had noted earlier (Rief and Hofmann, 2018). Leichsenring et al. (2018) opposed our arguments, based on other examples and their own trials advocating for psychodynamic treatments.First, we want to thank Leichsenring and colleagues for their thorough report and interest on our paper. The authors present an impressive variety of NIMs that have been used in prior publications, confirming our argument that there is no clear consensus for defining NIMs. We also accept their critique that we did not differentiate between non-inferiority and equivalence trials; our critique applies to both types and does not require this differentiation.The reasons for erroneous non-inferiority results of two treatments can be manifold. Poor study quality, insensitive ascertainment or statistical analyses procedures, poor implementation of (comparator) treatments are just a few examples, as outlined in our original commentary. Even if Leichsenring et al. do not appreciate our examples, they do not provide a convincing argument against these conclusions. We continue to encourage the scientific community to consider the pivotal question with regard to NIMs: What is the risk of erroneous equivalence conclusions that we want to accept as responsible clinicians and scientists in the era of the replicability crisis?We explicitly reject Leichsenring's notion that we misrepresented the results of the Steinert et al. (2017) study. Therefore, we present the original results in Fig. 1 here again. While the authors concluded that results indicate equality between two treatmen...

show abstract

“…In a reply to our comment on their article on non-inferiority testing (Leichsenring et al, 2018a;Rief and Hofmann, 2018b), Rief and Hofmann (2018a) reject our statement that they misinterpreted the results of the Steinert et al meta-analysis (Steinert et al, 2017). They maintain that a significant disadvantage of psychodynamic therapy compared with other therapies was shown.…”

mentioning

confidence: 87%