Statistics for Evaluating Pre-post Change: Relation Between Change in the Distribution Center and Change in the Individual Scores

Abstract

In a number of scientific fields, researchers need to assess whether a variable has changed between two time points. Average-based change statistics (ABC) such as Cohen's d or Hays' ω2 evaluate the change in the distributions' center, whereas Individual-based change statistics (IBC) such as the Standardized Individual Difference or the Reliable Change Index evaluate whether each case in the sample experienced a reliable change. Through an extensive simulation study we show that, contrary to what previous studies have speculated, ABC and IBC statistics are closely related. The relation can be assumed to be linear, and was found regardless of sample size, pre-post correlation, and shape of the scores' distribution, both in single group designs and in experimental designs with a control group. We encourage other researchers to use IBC statistics to evaluate their effect sizes because: (a) they allow the identification of cases that changed reliably; (b) they facilitate the interpretation and communication of results; and (c) they provide a straightforward evaluation of the magnitude of empirical effects while avoiding the problems of arbitrary general cutoffs.

Callous-Unemotional Traits Do Not Predict Functional Family Therapy Outcomes for Adolescents With Behavior Problems

“…To test whether CU traits were reduced over time in the context of FFT-treatment, both individual-centered and average-based change statistics were calculated ( Estrada et al, 2018 ). The reliable change index (RCI; Jacobson and Truax, 1991 ) was used as the individual-centered statistic and computed by dividing ICU12 change scores by the standard error of the difference between the two scores.…”

Erfolge der Rehabilitation bei chronischen nichtspezifischen Rückenschmerzen in Deutschland – Metaanalysen der Verläufe von Schmerzintensität und Funktionskapazität

“…Das einschlägigen Cochrane-Review [23] berichtet bei Vergleichen mit Wartegruppen für Schmerz ein d von 0,73, für Funktion von 0,49. Gemäß [18] bedeutet das, dass sich 23 bzw. 15 % durch Rehabilitation reliabel mehr verbessern als durch Abwarten.…”

“…Das sind Rehabilitanden, die sich so stark verbessert haben, dass die Wahrscheinlichkeit dieser Veränderung unter der Nullhypothese kleiner als 5 % ist. Dieser Reliable Change lässt sich nach Modellrechnungen[18] aus Cohens d bestimmen. Er wurde für jeden Vergleich berechnet und die Ergebnisse zu einem gewichteten Mittel zusammengefasst.…”

Statistical Modelling for Sports Scientists: Practical Introduction Using R (Part 1)

“…But on top of providing these distribution-based or groupbased estimates, one can provide expected uncertainties showing individual-based or response proportions as anchor-based (magnitude-based) estimates (Norman et al, 2001;Estrada, Ferrer & Pardo, 2019). It is important to note that these usually estimate the same information (Norman et al, 2001;Estrada, Ferrer & Pardo, 2019); e.g. the higher the Cohen's d the higher the proportion of higher responses.…”