The aim of this article is to present a set of evidence levels, accompanied by 14 quality or rigor questions, to foster a critical review of published single‐subject research articles. In developing these guidelines, we reviewed levels of evidence and quality/rigor criteria that are in wide use for group research designs, e.g. randomized controlled trials, such as those developed by the Treatment Outcomes Committee of the American Academy for Cerebral Palsy and Developmental Medicine. We also reviewed methodological articles on how to conduct and critically evaluate single‐subject research designs (SSRDs). We then subjected the quality questions to interrater agreement testing and refined them until acceptable agreement was reached. We recommend that these guidelines be implemented by clinical researchers who plan to conduct single‐subject research or who incorporate SSRD studies into systematic reviews, and by clinicians who aim to practise evidence‐based medicine and who wish to critically review pediatric single‐subject research.
Aim
To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews’ conclusions.
Method
A comprehensive search of 22 databases identified eligible systematic reviews with and without meta‐analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews‐2 (AMSTAR‐2) to appraise methodological quality.
Results
Eighty‐three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR‐2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR‐2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time.
Interpretation
Most of these systematic reviews are considered unreliable according to AMSTAR‐2. Current recommendations for treating children with CP based on these flawed systematic reviews need re‐evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high‐level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews.
Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP).
Quality issues in the sample were not limited to systematic reviews of non‐randomized trials, or to those about certain populations of CP or interventions.
The inclusion of meta‐analysis did not improve the level of confidence in these systematic reviews.
Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.