Study designs to detect sponsorship and other biases in systematic reviews

Dias, Sofia; Welton, Nicky J; Ades, A E

doi:10.1016/j.jclinepi.2010.01.005

Cited by 31 publications

(33 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, if we consider the GRADE domains “study limitations” and “publication bias,” it is likely that within the entire evidence ensemble, several studies will be vulnerable to precisely the same kinds of bias, operating to the same or similar extent, and in the same direction [28], [29], [30], [31], [32], [33]. We have also only considered that a single contrast can be biased.…”

Section: Discussionmentioning

confidence: 99%

A threshold analysis assessed the credibility of conclusions from network meta-analysis

Caldwell

Ades

Dias

et al. 2016

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

ObjectiveTo assess the reliability of treatment recommendations based on network meta-analysis (NMA).Study Design and SettingWe consider evidence in an NMA to be potentially biased. Taking each pairwise contrast in turn, we use a structured series of threshold analyses to ask: (1) “How large would the bias in this evidence base have to be before it changed our decision?” and (2) “If the decision changed, what is the new recommendation?” We illustrate the method via two NMAs in which a Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessment for NMAs has been implemented: weight loss and osteoporosis.ResultsFour of the weight-loss NMA estimates were assessed as “low” and six as “moderate” quality by GRADE; for osteoporosis, six were “low,” nine were “moderate,” and 1 was “high.” The threshold analysis suggests plausible bias in 3 of 10 estimates in the weight-loss network could have changed the treatment recommendation. For osteoporosis, plausible bias in 6 of 16 estimates could change the recommendation. There was no relation between plausible bias changing a treatment recommendation and the original GRADE assessments.ConclusionsReliability judgments on individual NMA contrasts do not help decision makers understand whether a treatment recommendation is reliable. Threshold analysis reveals whether the final recommendation is robust against plausible degrees of bias in the data.

show abstract

Section: Discussionmentioning

confidence: 99%

A threshold analysis assessed the credibility of conclusions from network meta-analysis

Caldwell

Ades

Dias

et al. 2016

Journal of Clinical Epidemiology

Self Cite

View full text Add to dashboard Cite

show abstract

“…77,96 Therefore, under the null hypothesis that the model adequately fits the data, D res would have a mean equal to the number of unconstrained data points for a perfectly fitted model. 77,96 The DIC is defined as:…”

Section: Assessing Model Fit and Deviancementioning

confidence: 99%

A cost-effectiveness modelling study of strategies to reduce risk of infection following primary hip replacement based on a systematic review

Graves

Wloch

Wilson

et al. 2016

Health Technol Assess

View full text Add to dashboard Cite

BackgroundA deep infection of the surgical site is reported in 0.7% of all cases of total hip arthroplasty (THA). This often leads to revision surgery that is invasive, painful and costly. A range of strategies is employed in NHS hospitals to reduce risk, yet no economic analysis has been undertaken to compare the value for money of competing prevention strategies.ObjectivesTo compare the costs and health benefits of strategies that reduce the risk of deep infection following THA in NHS hospitals. To make recommendations to decision-makers about the cost-effectiveness of the alternatives.DesignThe study comprised a systematic review and cost-effectiveness decision analysis.Setting77,321 patients who had a primary hip arthroplasty in NHS hospitals in 2012.InterventionsNine different treatment strategies including antibiotic prophylaxis, antibiotic-impregnated cement and ventilation systems used in the operating theatre.Main outcome measuresChange in the number of deep infections, change in the total costs and change in the total health benefits in quality-adjusted life-years (QALYs).Data sourcesLiterature searches using MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and the Cochrane Central Register of Controlled Trials were undertaken to cover the period 1966–2012 to identify infection prevention strategies. Relevant journals, conference proceedings and bibliographies of retrieved papers were hand-searched. Orthopaedic surgeons and infection prevention experts were also consulted.Review methodsEnglish-language papers only. The selection of evidence was by two independent reviewers. Studies were included if they were interventions that reported THA-related deep surgical site infection (SSI) as an outcome. Mixed-treatment comparisons were made to produce estimates of the relative effects of competing infection control strategies.ResultsTwelve studies, six randomised controlled trials and six observational studies, involving 123,788 total hip replacements (THRs) and nine infection control strategies, were identified. The quality of the evidence was judged against four categories developed by the National Institute for Health and Care ExcellenceMethods for Development of NICE Public Health Guidance(http://publications.nice.org.uk/methods-for-the-development-of-nice-public-health-guidance-third-edition-pmg4), accessed March 2012. All evidence was found to fit the two highest categories of 1 and 2. Nine competing infection control interventions [treatments (Ts) 1–9] were used in a cohort simulation model of 77,321 patients who had a primary THR in 2012. Predictions were made for cases of deep infection and total costs, and QALY outcomes. Compared with a baseline of T1 (no systemic antibiotics, plain cement and conventional ventilation) all other treatment strategies reduced risk. T6 was the most effective (systemic antibiotics, antibiotic-impregnated cement and conventional ventilation) and prevented a further 1481 cases of deep infection, and led to the largest annual cost savings and the greatest gains to QALYs. The additional uses of laminar airflow and body exhaust suits indicate higher costs and worse health outcomes.ConclusionsT6 is an optimal strategy for reducing the risk of SSI following THA. The other strategies that are commonly used among NHS hospitals lead to higher cost and worse QALY outcomes. Policy-makers, therefore, have an opportunity to save resources and improve health outcomes. The effects of laminar air flow and body exhaust suits might be further studied if policy-makers are to consider disinvesting in these technologies.LimitationsA wide range of evidence sources was synthesised and there is large uncertainty in the conclusions.FundingThe National Institute for Health Research Health Technology Assessment programme and the Queensland Health Quality Improvement and Enhancement Programme (grant number 2008001769).

show abstract

“…MTC relies on exactly the same assumptions as standard pairwise meta-analysis (i.e., choice and quality of the studies), although now these are applicable to the full set of interlinked trials. Therefore, the similarity between trials included in the network will also be a determinant of the internal validity of the analyses, at the risk of having a high confounding bias [49]. In the instances where direct and indirect evidence are combined for a particular comparison, it is also vital that there are no disagreements between the direct and indirect comparisons (for instance, in an MTC model comparing three treatments [e.g., A, B, and C], consistency is achieved when, for each pairwise comparison, no discrepancies can be found between the direct and indirect estimates of the parameter of interest [e.g., OR] derived from the model.…”

Section: Multiple Aggregate Data To Inform the Estimation Of Multiplementioning

confidence: 99%

Deriving Input Parameters for Cost-Effectiveness Modeling: Taxonomy of Data Types and Approaches to Their Statistical Synthesis

2012

View full text Add to dashboard Cite

show abstract

Study designs to detect sponsorship and other biases in systematic reviews

Cited by 31 publications

References 18 publications

A threshold analysis assessed the credibility of conclusions from network meta-analysis

A threshold analysis assessed the credibility of conclusions from network meta-analysis

A cost-effectiveness modelling study of strategies to reduce risk of infection following primary hip replacement based on a systematic review

Deriving Input Parameters for Cost-Effectiveness Modeling: Taxonomy of Data Types and Approaches to Their Statistical Synthesis

Contact Info

Product

Resources

About