A survey of publication practices of single‐case design researchers when treatments have small or large effects

Abstract

The published literature often underrepresents studies that do not find evidence for a treatment effect; this is often called publication bias. Literature reviews that fail to include such studies may overestimate the size of an effect. Only a few studies have examined publication bias in single-case design (SCD) research, but those studies suggest that publication bias may occur. This study surveyed SCD researchers about publication preferences in response to simulated SCD results that show a range of small to large effects. Results suggest that SCD researchers are more likely to submit manuscripts that show large effects for publication and are more likely to recommend acceptance of manuscripts that show large effects when they act as a reviewer. A nontrivial minority of SCD researchers (4% to 15%) would drop 1 or 2 cases from the study if the effect size is small and then submit for publication. This article ends with a discussion of implications for publication practices in SCD research.

A Multiple-Baseline Evaluation of Acceptance and Commitment Therapy Focused on Repetitive Negative Thinking for Comorbid Generalized Anxiety Disorder and Depression

“…Cuijpers et al, 2016;Bai et al, 2020). However, these findings should be taken with caution because, although the metric of the d-statistic computed in this study is the same as in group designs, the effect sizes found in SCED studies are usually higher than those found in group designs such as randomized controlled trials (Parker and Vannest, 2009;Shadish et al, 2016). Overall, this study indicates that RNT-focused ACT protocols are promising for the treatment of depression and GAD and warrant conducting randomized controlled trials to compare their effect versus waiting-list control conditions or brief versions of empirically established treatments such as behavioral activation or cognitive therapy.…”

Evaluation of a finger prompt variation in the treatment of pediatric feeding disorders

“…The authors did not report 5‐s acceptance which is typically the behavior researchers attempt to increase, thus the child may have never willingly opened the mouth. Finally, as with many single‐subject studies in this field, past reports have relied exclusively on retrospective collections of patients who received a similar treatment, raising important questions regarding the file drawer phenomenon (i.e., authors select studies with clear experimental effects to submit for publication and refrain from submitting work with insignificant effects, resulting in readers finding underrepresented negative results in the literature) as it related to treatment effectiveness (Shadish, Zelinsky, Vevea, & Kratochwill, ).…”

Open Science in the Field of Emotional and Behavioral Disorders

“…Publication bias and scarcity of studies with null findings also appear to exist in the single-case design (SCD) research base (Kittleman, Gion, Horner, Levin, & Kratochwill, 2018;Sham & Smith, 2014), which commonly informs policy and practice for students with EBD. In fact, Shadish, Zelinsky, Vevea, and Kratochwill (2016) found that a "non-trivial minority" (p. 656) of SCD researchers (4-15%, depending on the condition) reported they would drop one or two cases with small effect sizes before submitting a study for publication. Moreover, reviews of the special education literature suggest that direct replication studies are exceedingly rare (Makel et al, 2016), rendering many research findings unverified.…”

“…Preregistrations were designed primarily for group-research designs and are less commonly used for qualitative and SCD studies. As SCDs (a) are used to identify evidencebased practices for students with EBD (Kratochwill et al, 2012) and (b) appear to be subject to researcher bias (Shadish et al, 2016;Sham & Smith, 2014), preregistration may be relevant for SCD researchers as well. Some SCD researchers are beginning to preregister their studies, such as the preregistration by Sadler for an SCD study exploring the use of video self-modeling to reduce challenging behaviors in students with ASD (see https://www.sreereg.org/framework/pdf/index.php?id=1725).…”