Reporting and Interpreting Effect Sizes in Applied Health-Related Settings: The Case of Spirituality and Substance Abuse

Abstract

Inferential analysis using null hypothesis significance testing (NHST) allows accepting or rejecting a null hypothesis. Nevertheless, rejecting a null hypothesis and concluding there is a statistical effect does not provide a clue as to its practical relevance or magnitude. This process is key to assessing the effect size (ES) of significant results, be it using context (comparing the magnitude of the effect to similar studies or day-to-day effects) or statistical estimators, which also should be sufficiently interpreted. This is especially true in clinical settings, where decision-making affects patients’ lives. We carried out a systematic review for the years 2015 to 2020 utilizing Scopus, PubMed, and various ProQuest databases, searching for empirical research articles with inferential results linking spirituality to substance abuse outcomes. Out of the 19 studies selected, 11 (57.9%) reported no ES index, and 9 (47.4%) reported no interpretation of the magnitude or relevance of their findings. The results of this review, although limited to the area of substance abuse and spiritual interventions, are a cautionary tale for other research topics. Gauging and interpreting effect sizes contributes to a better understanding of the subject under scrutiny in any discipline.

Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments

“…In summary, authors are expected to systematically report and interpret ESs in studies using inferential statistics, and even more so in randomized, controlled experimental studies. However, based on our previous work in the clinical setting [52,53], we believe that we are not there yet.…”

“…Some studies have found that NHST p-values were rarely supplemented with information on ES, despite the existence of published guidelines [50,51]. In the field of substance abuse, a systematic review [52] revealed that 57.9% of the selected studies did not report ES indices, while 47.4% did not interpret the magnitude or relevance of their findings. Similarly, a review examining the effects of physical exercise on depressive symptoms [53] discovered that 52.4% of experimental studies did not report ES indices.…”