Evaluating Article Search and Selection Procedures in Special Education Literature Reviews

Abstract

Literature reviews allow education professionals to identify effective interventions and assess trends in research and practice. However, issues with transparency and the specific methods used in conducting literature searches have the potential to limit the utility of reviews. This study evaluated the description of article search procedures and selection processes appearing in literature reviews and meta-analyses published in special education journals from 2004 to 2016. Findings reveal a considerable increase in the number of published meta-analyses and systematic reviews over this time period. Nonetheless, the validity of reviews is potentially undermined by incomplete descriptions of methods. Reports of inclusion criteria further suggests the results of many reviews may be biased. A discussion of implications follows a description of findings.

Blending as a treatment for feeding disorders: A review of the literature

“…We selected the WWC (2017) quality indicators because of their emphasis on observable study features linked to internal validity. Using different quality standards may have produced different conclusions regarding the merit of the research (King et al, 2020a, 2020b). WWC indicators require each condition of a study to include a minimum of three data points per condition and a minimum of three replications of effect.…”

“…We also did not conduct a hand search of potential publication outlets. In employing three distinct search methods, our search included more identification methods than the majority of systematic reviews published in special education or behavior analysis (King et al, 2020a, 2020b).…”

“…Given the role of visual analysis in analyzing outcomes of single‐case design research, we excluded single‐case designs in which authors did not graphically display results (Manolov, Losada, Chacon‐Moscoso, & Sanduvete‐Chaves, 2016). Dissertations were included due to concerns regarding publication bias related to the exclusion of unpublished studies and the ability to control for study rigor through the application of quality indicators (Chitiyo, King, Krizon, Ablakwa, & Markelz, 2020; King, Davidson, Chitiyo, & Apple, 2020a). We excluded master's theses and other unindexed gray literature given the potential of partially included source material to bias results (Ferguson & Brannick, 2012).…”

“…Studies of insufficient methodological rigor invite alternative conclusions regarding the effect of an intervention (King et al, 2020a, 2020b). We evaluated identified articles using contemporary single‐case and group design standards of the WWC (2017).…”

Addressing Feeding Disorders Using High-Probability Sequencing for Children and Adolescents with Developmental Disabilities

“…Analyzing studies featuring nonconcurrent designs would likewise have influenced findings. We therefore encourage future inquiry into the influence of quality assessment on the results of syntheses, as the use of different selection criteria may result in different conclusions (King, Davidson, Chitiyo, & Apple, 2018). Regardless, we find a skeptical view of high-probability sequencing for food acceptance is warranted given (a) the few studies supporting its effectiveness in the absence of other procedures and (b) the cases that suggest high-probability sequencing is ineffective.…”