Methodological concerns for meta-analyses of meditation: Comment on Sedlmeier et al. (2012).

Abstract

We commend Sedlmeier et al. (2012) for their significant undertaking of meta-analysis of all meditation types on all psychological variables, but additional analyses may modify some of their conclusions. Whereas they suggest from visual inspection of funnel diagrams that there may be publication bias of underreporting low-effect studies on the Transcendental Meditation (TM) technique, quantitative tests do not indicate the presence of bias for any type of meditation. We additionally found that there was no significant difference in effect sizes between studies originating from researchers affiliated with a TM organization and studies from other universities. We found that comparison of different types of meditation on their global index was confounded because their global index aggregated different sets of variables for the different groups. That is, using composite indices that only aggregated variables for which each group had at least 3 studies confirmed the authors' conclusion that effect sizes for different research designs were not different, but found that effect sizes for the TM technique were significantly larger than effect sizes for mindfulness meditation or other meditations. We also located 35 studies on the TM technique that appear to meet the authors' inclusion criteria that were missed by their meta-analysis, and several others on important psychosocial behavioral variables, such as job performance, substance abuse, and prison recidivism that were not reviewed. In addition, we suggest that future meta-analyses on psychological variables include cross-validating physiological studies.

What contributes to wellbeing gains – proficiency or duration of meditation related practices?

“…The results show that BKRY practitioners who adapt to the lifestyle changes experience a significant sense of progress in wellbeing. This is in line with other reports that show that meditation practitioners experience enhanced wellbeing on a number of measures (Manocha et al, 2012;Orme-Johnson & Dillbeck, 2014).…”

Developing a Small Theory of Treatment of Yoga

“…Due to the fact that yoga is a holistic way of life, the research method for yoga is always going to be weak. Along this line, a recent article on meditation research indicated that [yoga] research might be better served when it is driven by theory [34].…”

Meta-analyses and other methodological issues in meditation research: Reply to Orme-Johnson and Dillbeck (2014).

“…We calculated the mean weighted (by sample size) effect size of these three studies and found r ϭ .17, that is, an effect that is smaller than the average effect size we found in our original analysis for TM studies (r ϭ .27). Strictly speaking, Orme-Johnson and Dillbeck (2014) were right to contend that our coverage of TM studies was not complete. Indeed, meditation research has been performed by a very heterogeneous group of researchers and scattered across a huge range of journals.…”

“…Even the quite heterogeneous group of other meditation techniques sometimes scored highest, as for the variable we termed cognition. Orme-Johnson and Dillbeck (2014) have argued that these empirically found differences are inconsistent with the assumption of a uniform effect of meditation. They are very probably right.…”

“…

We are grateful to Orme-Johnson and Dillbeck (2014) for raising several potential methodological concerns they have with our meta-analysis of the psychological effects of meditation and for thereby providing us with the opportunity to clarify these points. Orme-Johnson and Dillbeck raised 4 points that they believed might have led to unfair treatment of studies that reported the effects of transcendental meditation (TM).

…”