Equivalence-based measures of clinical significance: assessing treatments for depression

Abstract

Treatment efficacy is largely determined by statistical significance testing, and clinical significance testing is often used to quantify or qualify the efficacy of a treatment at the individual or group level. This study applies the equivalence-based clinical significance model proposed by Kendall, Marrs-Garcia, Nath, and Sheldrick (1999) and a revised model proposed by Cribbie and Arpin-Cribbie (2009) to the assessment of treatments for depression. Using several studies that investigated treatments for depression, the authors tested whether the posttreatment means were equivalent to those for a similar normal comparison group. All of the studies had significant improvement from pretest to posttest, although for many of the studies the treated group was not equivalent to a normal comparison group at posttest. Further, there are important differences between the conclusions drawn from the Kendall et al. and Cribbie and Arpin-Cribbie methods for assessing equivalence-based clinical significance.

Assessing clinical significance using robust normative comparisons

“…Therefore, although a given intervention may render a statistically significant difference from pre-to post-test data for the treatment group, the intervention does not necessarily return the group to a state of normal functioning. As Nasiakos et al (2010) mention, methods that determine clinical significance, such as normative comparisons, provide supplementary information regarding whether the treated group returned to a state of "normal" functioning, a common goal of intervention studies for depression. By conducting robust normative comparisons on real data, this study demonstrates that different conclusions indeed do arise when clinical significance tests are conducted as opposed to statistical significance tests.…”

“…Although there are a number of different alternatives that could have been considered, depression was selected because of its prevalence and the fact that a vast amount of research has been published on the effectiveness of existing therapies (e.g., Driessen et al, 2007;Gaynor et al, 2003;Nasiakos, Cribbie, & Arpin-Cribbie, 2010). It is important to point out though, as highlighted by a reviewer of the paper, that different depression scales are sensitive to different forms, symptoms, and severities of depression and therefore caution is needed in interpreting the results.…”

“…As Nasiakos et al (2010) mention, in order to determine clinical significance, treatment groups Table I. Description of the intervention studies.…”

Changes in Trauma‐Related Emotions Following Treatment With Dialectical Behavior Therapy for Posttraumatic Stress Disorder After Childhood Abuse

“…Approaches proposed for comparing patients’ scores with nonclinical values are either exclusively based on values observed in the nonclinical reference group or rely on relative approaches based on values in the patient and nonclinical reference groups (for a review, see Bauer, Lambert, & Nielsen, ; Nasiakos, Cribbie, & Arpin‐Cribbie, ). In the first instance, an equivalence range (e.g., a multiple of the standard deviation around the mean in the nonclinical population) is defined.…”

Therapeutic alliance and different treatment formats when delivering internet-based CBT for depression

“…Most studies have defined successful treatment of depression by means of statistical significance, but researchers have called for the use of clinical significance as a more reliable measure of meaningful individual change (Nasiakos, Cribbie, & Arpin-Cribbie, 2010). The problem with clinical significance so far has been that the definitions, statistical methods and instruments used for measuring have varied greatly, although specific methods as the Jacobson method has been put forward as preferable methods calculating clinical significance (Bauer, Lambert, & Nielsen, 2004;Jacobson & Truax, 1991).…”