Clinical Significance Methods: A Comparison of Statistical Techniques

Abstract

Clinically significant change refers to meaningful change in individual patient functioning during psychotherapy. Following the operational definition of clinically significant change offered by Jacobson, Follette, and Revenstorf (1984), several alternatives have been proposed because they were thought to be either more accurate or more sensitive to detecting meaningful change. In this study, we compared five methods using a sample of 386 outpatients who underwent treatment in routine clinical practice. Differences were found between methods, suggesting that the statistical method used to calculate clinical significance has an effect on estimates of meaningful change. The Jacobson method (Jacobson & Truax, 1991) provided a moderate estimate of treatment effects and was recommended for use in outcome studies and research on clinically significant change, but future research is needed to validate this statistical method.

Evaluation of a Multi-disciplinary Back Pain Rehabilitation Programme—Individual and Group Perspectives

“…Indeed if samples are large enough, quite small group-level changes may be statistically significant, but essentially meaningless in clinical terms. Whilst different techniques will provide different results [31], there does appear to be a reasonable comparability between approaches [38,39] which is supportive of the construct as a whole.…”

“…A number of methods are available to assess statistically significant individual change [30] and clinical significance/meaningful change [31]. Ferguson et al [19] describe the use of Reliable Change Index with SF-36 data and the approach they used has been adopted within this study, but using relevant UK norm data [32,33].…”

A secondary analysis of internet-delivered cognitive-behaviour treatment for adults with severe depressive symptoms (Preprint)

“…Reliable change is an important concept in clinical investigations due to its potential to provide a complete picture of the impact of interventions, which goes beyond the averages of the group and outlines change at an individual level [37]. Reliable change analysis in research completers illustrated 21 individuals reliably improved, with remainder being unchanged.…”

Future of Psychometrics: Ask What Psychometrics Can Do for Psychology

“…Bauer, Lambert, and Nielsen (2004) compared five statistics for individual-change assessment that address the latter two problems and argued in favor the Jacobson-Truax method because it is easy to compute, clinical cutoff scores are available for many widely known instruments, and the method moderates some of the extreme effects of other methods. Atkins, Bedics, McClinchey, and Beauchaine (2005) drew a similar conclusion.…”