Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes.

Abstract

In primary studies, psychotherapy researchers frequently search for mediator and moderator variables that can help them understand the relationship between treatment and outcome. Yet a review of past psychotherapy meta-analyses revealed that none examined the possible role of mediator variables; and although all of them searched for moderators of study outcome, that search was generally not as complete as it could have been. This article illustrates methods for studying such mediator and moderator variables in meta-analysis, discusses their advantages and disadvantages, and shows how the inclusion of these variables can change interpretation of meta-analytic results. In particular, the perennial interpretation of past psychotherapy meta-analyses that therapeutic orientation makes no difference to outcome--or as the dodo bird put it: "Everyone has won and all must have prizes'--may be wrong. Orientation may make significant difference, but only by virtue of its moderating and mediating effects.To the best of our knowledge, all meta-analyses ever done have concluded that (on the average) clients receiving psychotherapy do better than clients not receiving psychotherapy. In fact, the computation of average therapy effects over studies is the defining strength of meta-analysis. But this strength leads to a criticism of meta-analysis: Knowledge of average effects says nothing about when, where, why, and how therapy works. The latter questions concern mediators and moderators of therapy outcome. The present article describes methods for addressing such questions in meta-analysis.Moderators and mediators are third variables that help researchers to understand the relationship between independent and dependent variables (Baron & Kenny, 1986). "A moderator is a qualitative (e.g., sex, race, class) or quantitative (e.g., level of reward) variable that affects the direction and/or strength of the relations between an independent or predictor variable and a dependent or criterion variable" (Baron & Kenny, 1986, p. 1174. Moderators cause statistical interactions. Some moderator variables are categorical. Suppose, for example, that behavioral therapies yielded high effect sizes on behavioral presenting problems but low effect sizes on nonbehavioral presenting problems, with the opposite pattern emerging for nonbehav-

Psychological treatment of postpartum depression: a meta‐analysis

“…The finding that different types of psychological interventions are about equally effective in the treatment of depression also was noted in other meta-analyses (Churchill et al, 2001;Cuijpers et al, in press;Wampold, Minami, Baskin, & Callen Tierney, 2002). In the general psychotherapy literature, it has been debated for more than three decades now whether psychotherapies are actually equally effective (Cuijpers, 1998;Luborsky, 1995;Luborsky, Singer, & Luborsky, 1975;Shadish & Sweeney, 1991), but no definite answer has been found. It is possible that most effects of psychological treatments are caused by common, nonspecific factors and not by specific techniques, but it also is possible that the effects of psychotherapy are realized by different, therapy-specific mechanisms (Butler & Strupp, 1986) and that the number of possible mediators and moderators is so large that small differences between treatments in specific groups of patients remain unnoticed.…”

Extracting and converting quantitative data into human error probabilities

“…On the other hand, in mediation analysis, it has been recommended to use structural equation modeling. In fact, although moderation analysis can be examined with the above statistics, researchers have supported the structural equation modeling approach for both moderators and mediators [4]. Nevertheless, for simplicity, chi-square, ANOVA, and regression analysis are suitable statistics in examining moderators.…”

“…A moderator is a third variable that can affect the direction and/or strength between the primary variable of interest and the outcome measures. Thus, in terms of statistics, moderators lead to interactions [4]. For example, stress may have a relatively strong effect on performance during high workload but relatively weak effect during low workload.…”

Investigation of the Phase Model of Psychotherapy Across Therapeutic Orientations: Are Different Approaches Actually All That Different?

“…The controversy centers on a conclusion which has been dubbed the dodo bird verdict (Messer & Wampold, 2002) and states that all psychotherapy outcome rates are similar; there is little difference between the various forms of psychotherapy since they all seem to have virtually the same level of effect as supported by meta-analytic findings. A number of researchers have criticized the dodo bird verdict on the basis of meta-analytic approaches obscuring the details of the impact of each technique by reducing them to a single number (an effect size) which tells us little or nothing of the real world impact of these different treatment styles (e.g., Beutler, 2002;Chambless, 2002) Additionally, there has been criticism which has centered on difficulties with the methodology of meta-analysis, specifically that most authors do not perform appropriate moderator or mediator analyses and thus often neglect important third variables that likely exert an impact on the effectiveness of differing types of psychotherapy (Shadish & Sweeney, 1991). …”