Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

Abstract

Objective To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. Design Combined analysis of data from three metaepidemiological studies based on collections of metaanalyses. Data sources 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or nonblinded trials exaggerate intervention effect estimates. Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and nondrug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. Conclusions The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II)

“…We used central randomisation stratifying for important predictive factors,51 52 53 54 used blinded outcome assessors,51 52 53 54 assessed several register based outcomes that are likely less influenced by knowledge about intervention group affiliation than other outcomes, included stratification factors in our main analyses,55 56 conducted our main analyses on the data where missingness was controlled by multiple imputation,57 58 conducted our analyses blinded for intervention group,59 and drew our conclusions without knowledge of intervention group 59. Moreover, although we based our sample size calculation on a power of 80%, by inflating our sample by 30% and analysing all participants using multiple imputations, we actually had a power of 91% to detect the a priori defined least significant difference regarding the primary outcome.…”

Self‐help educational booklets for the prevention of smoking relapse following smoking cessation treatment: a randomized controlled trial

“…Lack of blinding is unlikely to influence the objectively measured primary outcome 22, although bias could be introduced in the self‐reported measurement of process variables such as reported receiving and reading of booklets. Possible recall bias is unlikely to be a major concern for the validity of estimated treatment effect, but it could affect analyses of process and mediating variables.…”

Concomitant Atrial Fibrillation Surgery for People Undergoing Cardiac Surgery

“…Data synthesis-We undertook meta-analyses only if the treatments, participants, and the underlying clinical questions in the studies were similar enough for pooling to be appropriate (Wood 2008). If an I 2 was less than or equal to 50%, then we used a fixed-effect model, whereas if the I 2 was greater than 50%, then we used a random-effects model (Higgins 2011).…”