Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions

Saltaji, Humam; Armijo‐Olivo, Susan; Cummings, Greta G.; Amin, Maryam; Costa, Bruno R. da; Flores‐Mir, Carlos

doi:10.1186/s12874-018-0491-0

Cited by 69 publications

(53 citation statements)

References 63 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result is in line other meta‐analyses within medical (Saltaji et al. ) and biological (van Wilgenburg and Elgar , Holman et al. ) sciences, in which non‐blind studies generally yield higher effect sizes when compared with studies using blinding protocols.…”

Section: Discussionsupporting

confidence: 86%

“…, Saltaji et al. ). Our meta‐analysis of spatial patterns in herbivory not only confirmed this tendency, but it also uncovered other sources of cognitive biases in biological research (Table ).…”

Section: Discussionmentioning

confidence: 99%

“…, Saltaji et al. ). At the same time, blinding is only rarely reported in ecological and evolutionary studies (Forstmeier et al.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Biases in studies of spatial patterns in insect herbivory

Zvereva

Kozlov

2019

Ecological Monographs

View full text Add to dashboard Cite

The properties of the human mind are responsible for a number of biases that affect the quality of scientific research. However, scientists working in the fields of ecology and environmental science rarely take these biases into account. We conducted a meta‐analysis of data extracted from 125 publications comparing woody plant damage by defoliating insects in different environments in order to understand the extent to which our knowledge on spatial patterns in herbivory is affected by various biases. We asked which research methods are most prone to biases and whether these biases lead to overestimation of the effects under study. The effect sizes (ESs) decreased with increases in the numbers of plant species involved in the study, with 61% lower ESs for herbivory estimated on all plants growing in study plots compared to herbivory on selected species. ESs also depended on the leaf sampling procedure: when all leaves from a tree or branch were sampled for measurements of herbivory or when random or systematic selection protocols were applied, ESs were 74% smaller than in cases of more subjective haphazard selection. In addition, ESs were 97% and 135% greater when the person conducting sampling and measuring was aware of the research hypothesis or sample origin, when compared with situations when the observer was blinded to these factors. The impacts of cognitive biases on the study outcomes significantly decreased with the increase in publication year; however, this pattern emerged mostly due to high‐ranked journals and was non‐significant for other journals. Using the studies of spatial patterns in herbivory as an example, we showed that our ecological and environmental knowledge is considerably biased due to an unconscious tendency of researchers to find support for their hypotheses and expectations, which generally leads to overestimation of the effects under study. Cognitive biases can be avoided by using different methods, such as applying randomization procedures in sampling and blinding of research hypotheses and sample origins. These measures should be seen as obligatory; otherwise, accumulation of the biased results in primary studies may ultimately lead to false general conclusions in subsequent research synthesis.

show abstract

Section: Discussionsupporting

confidence: 86%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Biases in studies of spatial patterns in insect herbivory

Zvereva

Kozlov

2019

Ecological Monographs

View full text Add to dashboard Cite

show abstract

“…In all instances, blinding had surprisingly little effect 723. Two additional recent studies partly confirmed this pattern: an analysis of physiotherapy trials24 found little evidence of an impact of blinding of patients or of outcome assessors, and a study of oral health trials25 found no evidence of an impact of blinding of outcome assessors, though some evidence of a moderate effect of patient blinding.…”

Section: Discussionmentioning

confidence: 88%

Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study

Moustgaard

Clayton

Jones

et al. 2020

BMJ

178

188

View full text Add to dashboard Cite

Objectives To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). Design Meta-epidemiological study. Data source Cochrane Database of Systematic Reviews (2013-14). Eligibility criteria for selecting studies Meta-analyses with both blinded and non-blinded trials on any topic. Review methods Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. Results The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. Conclusion No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

show abstract

“…It aims to prevent bias associated with expectations from patients, investigators, and assessors. Treatment effects may be overestimated or underestimated without blinding [ 25 , 26 ]. Five trials (3.47%) in this study reported the blinding methods, and three of six trials (50%) described the similarities of interventions.…”

Section: Discussionmentioning

confidence: 99%

Reporting Quality of Randomized Controlled Trials for the Treatment of Eczema with Chinese Patent Medicine Based on the CONSORT‐CHM Formulas 2017

Zhou

et al. 2020

Evidence-Based Complementary and Alternative Medicine

View full text Add to dashboard Cite

Objective. Chinese patent medicine (CPM) has been widely used to treat eczema in mainland China for decades. This study aims to investigate circulating CPM for eczema in mainland China and to evaluate the reporting quality of randomized controlled trials (RCTs) of them by using the CONSORT-CHM formulas 2017 (Consolidated Standards of Reporting Trials for Chinese herbal medicine formulas 2017). Methods. Circulating CPM with the indication for eczema was selected by searching three drug databases and confirmed by contacting the manufacturers. RCTs for the treatment of eczema with CPM were selected in four Chinese literature databases and four English literature databases from their inception to August 31, 2019. The reporting quality of included RCTs was assessed based on the CONSORT-CHM formulas 2017. A univariate analysis was conducted to identify the factors associated with the reporting quality. Results. A total of 70 circulating CPMs had the indication for eczema. Among them, 21 CPMs with 144 RCTs reached the eligible criteria. The mean overall quality score (OQS) of 144 RCTs was 19.85 ± 2.73, which was much less than the maximum score of 38. Of the 38 items, 12 items were reported in over 70% of the trials, 6 items were reported in 50%–70% of the trials, and 16 items were reported in less than 50% of the trials. Publication after 2015 (P<0.001) and the first author from a university hospital (P=0.010) were associated with the better reporting quality. Conclusion. There are a lot of circulating CPMs with the indication for eczema in mainland China, but both the quantity and the reporting quality of RCTs regarding those CPMs are suboptimal. It is necessary that authors and journal editors learn and adhere to the CONSORT-CHM formulas 2017 to enhance the reporting quality of RCTs for the treatment of eczema with CPM.

show abstract

Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions

Cited by 69 publications

References 63 publications

Biases in studies of spatial patterns in insect herbivory

Biases in studies of spatial patterns in insect herbivory

Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study

Reporting Quality of Randomized Controlled Trials for the Treatment of Eczema with Chinese Patent Medicine Based on the CONSORT‐CHM Formulas 2017

Contact Info

Product

Resources

About