Background An important part of the systematic review methodology is appraisal of the risk of bias in included studies. Cochrane systematic reviews are considered golden standard regarding systematic review methodology, but Cochrane’s instructions for assessing risk of attrition bias are vague, which may lead to inconsistencies in authors’ assessments. The aim of this study was to analyze consistency of judgments and support for judgments of attrition bias in Cochrane reviews of interventions published in the Cochrane Database of Systematic Reviews (CDSR). Methods We analyzed Cochrane reviews published from July 2015 to June 2016 in the CDSR. We extracted data on number of included trials, judgment of attrition risk of bias for each included trial (low, unclear or high) and accompanying support for the judgment (supporting explanation). We also assessed how many Cochrane reviews had different judgments for the same supporting explanations. Results In the main analysis we included 10,292 judgments and supporting explanations for attrition bias from 729 Cochrane reviews. We categorized supporting explanations for those judgments into four categories and we found that most of the supporting explanations were unclear. Numerical indicators for percent of attrition, as well as statistics related to attrition were judged very differently. One third of Cochrane review authors had more than one category of supporting explanation; some had up to four different categories. Inconsistencies were found even with the number of judgments, names of risk of bias domains and different judgments for the same supporting explanations in the same Cochrane review. Conclusion We found very high inconsistency in methods of appraising risk of attrition bias in recent Cochrane reviews. Systematic review authors need clear guidance about different categories they should assess and judgments for those explanations. Clear instructions about appraising risk of attrition bias will improve reliability of the Cochrane’s risk of bias tool, help authors in making decisions about risk of bias and help in making reliable decisions in healthcare. Electronic supplementary material The online version of this article (10.1186/s12874-019-0717-9) contains supplementary material, which is available to authorized users.

It is challenging to keep systematic reviews (SR) current and updated. Cochrane designated some of its SRs as "stable," that is, not in need of updating. The issue of stabilizing an SR is an important in research synthesis, because it could help reduce research waste. The aim of this study was to analyze publicly available justifications for stabilizing a Cochrane review, with the ultimate goal of helping to make decisions about whether the update of any SR is warranted. Methods: We analyzed Cochrane reviews labeled as stable in Archie, Cochrane's system for managing the editorial/publishing process. From the "What's new" section of the reviews in the Cochrane Library, we extracted justification for stabilization. Results: We included 545 Cochrane reviews labeled in Archie as stable on October 28, 2019. The most common of the five reasons for stabilization was that "last search did not identify any potentially relevant studies likely to change conclusions" (N = 99; 18%), followed by "research area no longer active" (N = 86; 16%), "review is or will be superseded" (N = 41; 7.5%), "evidence is conclusive" (N=35; 6.4%), and "intervention no longer in general use" (N = 34; 6.2%). For the 269 (49%) Cochrane reviews, we considered that the justification for stabilization was not clearly described, that is, sufficiently informative. Conclusions: Cochrane reviews would benefit from more transparency and consistency in publicly available justifications for stabilizing reviews. Further work in this field will help make decisions about the futility of further research and deciding on enough evidence in the field of research synthesis.

No abstract

Background Clinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the ‘other bias’ domain of Cochrane RoB tool. Methods We analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding ‘other bias’ from the RoB table and accompanying support for the judgment. We categorized different types of other bias. Results We analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had ‘other bias’ domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews. Conclusion Cochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews. Electronic supplementary material The online version of this article (10.1186/s12874-019-0718-8) contains supplementary material, which is available to authorized users.

Background and objective We explored how systematic reviews evaluated paracetamol and ibuprofen for treating pain in children, as these two non‐opioid analgesics are well‐established medicines included in most national essential medicines lists. Databases and data treatment We carried out an overview of systematic reviews (SRs) of randomized controlled trials (RCTs) of interventions (PROSPERO registration: 42016045367). We searched MEDLINE, EMBASE, Cochrane Database of Systematic Reviews (CDSR) and Database of Reviews of Effects (DARE) up to 23 August 2017. We used AMSTAR checklist to analyse methodological quality of included SRs. Results We found 17 SRs with 72 unique RCTs; the majority of those trials included under 100 children. Positive conclusive evidence was found in only one SR, regarding safety of paracetamol. Conclusions of other SRs for efficacy and safety of ibuprofen and paracetamol were inconclusive, unclear, or there was no opinion. Only one SR analysed efficacy of ibuprofen and other non‐steroidal anti‐inflammatory drugs (NSAIDs) in chronic pain and the conclusion was that there was no evidence from RCTs that NSAIDs were effective for chronic non‐cancer pain in children and adolescents. Most of the SRs addressed very narrow questions, included few trials, with few children and were of low or medium methodological quality. Conclusions Most SRs on two relevant medicines have inconsistent conclusions and doubt upon their effectiveness. Instead of focusing on very narrow questions, SRs should examine more comprehensive research topics to obtain a general sense of consistency, particularly when analysing established medicines. Significance Evidence behind two analgesics—ibuprofen and paracetamol—that are well‐established medicines for children in most countries appears limited, judging by the systematic reviews. The discrepancy between clinical use and the extensive evidence we reviewed may be a result of the selective criteria in the reviews examined. We need new, and better evidence syntheses supporting the use of these two medicines in wide indications regarding pain in children.

Some Cochrane reviews were declared stable or closed, that is, not in need of updating. For some of them, it has been declared that conclusions will not (or it is unlikely they will) change with further studies. We explored whether there is a discernable decision-making pattern for decisions about the conclusiveness and stabilization of these reviews. We analyzed Cochrane reviews published until April 2020 labeled as stable or closed. We extracted the rationale leading to the decision declaring that the conclusion is not expected to change with further studies. Furthermore, we assessed whether the reviews used GRADE analysis. We extracted data from summary of findings (SoF) tables on the direction of effect, statistical significance, and I 2 values for the first and primary outcomes in SoFs, conclusions in the abstract and review, and implications for practice and future research. We included 40 stable/closed Cochrane reviews. Rationales for their stabilization did not enable any insight into the Cochrane's decision-making algorithm for considering the evidence as conclusive. Among 191 outcomes presented in the SoFs, 70% were rated with either low or very low certainty evidence. None of the reviews mentioned in the text that the review should be stabilized or closed, or that there is sufficient evidence on the subject. Reasons for stabilizing/closing Cochrane reviews were unclear, and we could not discern any pattern of "conclusive review" traits. Definition of systematic review conclusiveness is still lacking, which may contribute to research waste.

It is necessary to raise awareness of the unacceptability of such pressures and interventions in relation to examination results, and to change the current professional culture in which the imposition of such pressures by students, their parents and staff colleagues is considered acceptable and common.

No abstract