A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study

Tricco, Andrea C.; Cogo, Elise; Page, Matthew J.; Polisena, Julie; Booth, Alison L.; Dwan, Kerry; MacDonald, Heather; Clifford, Tammy; Stewart, Lesley; Straus, Sharon E.; Moher, David

doi:10.1016/j.jclinepi.2016.03.025

Cited by 62 publications

(62 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recent study compared 80 non-Cochrane systematic reviews with their published protocols, and found that almost all (92.5%) differed from their protocols in at least one of the methods-related “Preferred reporting items for systematic review and meta-analysis protocols” (PRISMA-P) items and their subcategories, with only 7% of these providing an explanation [14]. Similar discrepancies between non-Cochrane systematic reviews and their PROSPERO protocols have been reported [15–18]. Although these discrepancies appear to be fairly common, reasons for the differences are rarely addressed.…”

Section: Discussionmentioning

confidence: 71%

Assessment of whether published non-Cochrane systematic reviews of nursing follow the review protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO): A comparative study

Mei

Gao

et al. 2020

Preprint

View full text Add to dashboard Cite

23Purpose: We compare published non-Cochrane reviews of nursing with their pre-registered 24 protocols on PROSPERO to quantify the prevalence of differences and the extent to which the 25 differences were explained. 26Methods: We searched for protocols and their corresponding reviews in PROSPERO's nursing 27 group that were "completed and published" from inception to September 8 th , 2019. Two 28 authors independently identified differences and classified the difference as none, partial, or 29 complete, and determined if the existed differences had been explained. Frequency (n), 30 percentage (%), median, and inter-quartile ranges were used to analyze the extent of differences 31 and explanations. 32 Results:We identified 22 pre-registered protocols and their reviews. All 22 pairs (100%) 33 exhibited differences. Eighteen pairs (82%) showed differences in at least six methodological 34 sections, while 21 pairs (95%) involved completed difference in at least one section. The 35 median number of differences per review was 8.00 (upper quartile = 6.00, lower quartile = 36 9.75). The differences involved all 13 compared methodology-related sections. Only 5 (3%) of 37 all differences were explained in the systematic reviews. 38 Conclusions:We observed widespread differences between non-Cochrane reviews of nursing 39 and their protocols recorded in PROSPERO, with relatively few explanations for the changes. 40Measures including establishing a new item in the reporting guideline of systematic reviews to 41 guide reporting and explaining the reasons for differences between protocols and systematic 42 reviews or even requiring authors to do so at the Journal's author guideline are recommended 43 to improve transparency. 44 45 3 46 126 protocol or acknowledgment and explanation of the alterations in the published review.130Differences were judged and classified using an internal guideline that was developed, 131 independently pilot-tested (in n=10 pairs), and revised by two authors (K.Y.H. and B.M.) (S1 132 Appendix). The categories used were: no difference (when the systematic review and protocol 133 matched or when a section was either not applicable or not reported), partial difference (when 134 a section was either not fully specified in the systematic review or the protocol, or when minor 135 differences were observed between parts of sections) , and complete difference (when aspects 136 of a section were modified in the systematic review or when a section was omitted from either 137 the protocol or the review). If judgments differed on a section, we chose the ''worst case'' 138 (e.g., when we observed a partial and a complete difference, we classified the compared section 139 as having a complete difference). If there were more than two partial differences concerning 140 the one section, we also classified it as having a complete difference. 141 Statistical methods 142 Statistical analysis was performed with Microsoft Excel 2016 software. Frequency (n), 143 percentage (%), median and Inter-quartile range were used ...

show abstract

Section: Discussionmentioning

confidence: 71%

Assessment of whether published non-Cochrane systematic reviews of nursing follow the review protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO): A comparative study

Mei

Gao

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…64% of reviews (n=25) exhibited changes in the "outcome" section. When authors make post-protocol modifications to review outcomes (that is, addition, removal, or re-prioritization) based on significance of outcome in the completed review, it can introduce bias into the review process, mislead readers and possibly affect patient care [60][61][62][63][64][65]. Moreover, through awareness of all study characteristics, systematic reviewers might be able to drive the results in different ways regarding participants (e.g., by applying age limits or rules when not all patients in a study met the inclusion criteria), intervention and comparison (e.g., omission of comparators which show an intervention less effective than the retained comparators), and definition of study designs (e.g., broadening of the types of studies included beyond randomized controlled trials alone to other potentially less-robust forms of comparison).…”

Section: The Categories and Impact Of Changesmentioning

confidence: 99%

“…(PRISMA-P) items and their subcategories, while a mere 7% provided an explanation [67]. Additionally, an existing study investigated difference between non-Cochrane systematic reviews and their PROSPERO records and found that a third of systematic reviews changed or did not specify the primary outcome [63]. The existing researches showed that the differences between the published non-Cochrane reviews and their protocols were quite common.…”

Section: Improving the Transparency And Its Implementationmentioning

confidence: 99%

Inconsistencies are common between non-Cochrane systematic reviews and their protocols registered in the International prospective register of systematic reviews (PROSPERO) but were seldom explained

Hu¹,

Zhang²,

Zhang³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Protocols of systematic reviews allow for planning and documentation of review methods and thus improve the transparency of the reviews process. However, pre-registration of a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online database for the registration of non-Cochrane systematic reviews. The purpose of this study is to compare published non-Cochrane reviews with their pre-registered protocols on PROSPERO to determine what changes, if any, have been made, and how likely these changes are to impact the quality of systematic review. Methods: This is a retrospective comparative study. We searched for protocols on PROSPERO platform that were registered in 2018 and then selected the protocols that full text have been published as of January 1st 2019. Published full texts were identified through the protocol's final publication citation. Two authors independently compared and identified changes between protocols and systematic reviews and then evaluated the impact (improve, reduce, or unclear) of these changes on the reporting or methodology quality of reviews. Descriptive statistics of percentage (%) and frequency (n) were conducted. Results: We identified 39 pairs, all of which exhibited changes. “Search strategy”(92%, n=36), “data extraction”(90%, n=35), “data synthesis”(77%, n=30), “outcome”(64%, n=24), and “subgroup analysis”(64%, n=24) all showed significant changes. All changes to only one review were considered to improve the reporting or methodology quality, and the remaining 97% of reviews (n=38) contained changes that were considered to reduce the methodology or reporting quality or that had an unclear impact on systematic reviews. Conclusions: Changes between the non-Cochrane systematic reviews and their protocols recorded on PROSPERO were widespread. Some of the changes reduced the methodology or reporting quality of systematic reviews or had an unclear impact. Measures should be taken to further improve the transparency of the non-Cochrane systematic reviews. Adding a new item in updated “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) and “Meta-analysis of Observational Studies in Epidemiology” (MOOSE) to guide reporting and explaining the changes, as well as advising peer reviewers (and editors) to check the reviews against the protocols are two suggested fundamental solutions.

show abstract

“…Journal editors and systematic reviewers therefore need to be mindful of whether the reported outcomes and exact analyses were selected prior to data analysis . Regrettably, it is not uncommon for even primary outcomes to change between study planning and completion, potentially undermining the integrity of the study . Reasons for, and timing of, any changes to planned reporting should be sought from trial authors.…”

Section: Declaration Of Interestmentioning

confidence: 99%

Reducing research waste in benign gynaecology and fertility research

Duffy

Bhattacharya

Herman

et al. 2017

BJOG

View full text Add to dashboard Cite

A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study

Abstract: We recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.

Cited by 62 publications

References 12 publications

Assessment of whether published non-Cochrane systematic reviews of nursing follow the review protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO): A comparative study

Assessment of whether published non-Cochrane systematic reviews of nursing follow the review protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO): A comparative study

Inconsistencies are common between non-Cochrane systematic reviews and their protocols registered in the International prospective register of systematic reviews (PROSPERO) but were seldom explained

Reducing research waste in benign gynaecology and fertility research

Contact Info

Product

Resources

About