Impact of Selection Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions: A Meta-epidemiological Study

Abstract

Emerging evidence suggests that design flaws of randomized controlled trials can result in over- or underestimation of the treatment effect size (ES). The objective of this study was to examine associations between treatment ES estimates and adequacy of sequence generation, allocation concealment, and baseline comparability among a sample of oral health randomized controlled trials. For our analysis, we selected all meta-analyses that included a minimum of 5 oral health randomized controlled trials and used continuous outcomes. We extracted data, in duplicate, related to items of selection bias (sequence generation, allocation concealment, and baseline comparability) in the Cochrane Risk of Bias tool. Using a 2-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity, we quantified the impact of selection bias on the magnitude of ES estimates. We identified 64 meta-analyses, including 540 randomized controlled trials analyzing 137,957 patients. Sequence generation was judged to be adequate (at low risk of bias) in 32% ( n = 173) of trials, and baseline comparability was judged to be adequate in 77.8% of trials. Allocation concealment was unclear in the majority of trials ( n = 458, 84.8%). We identified significantly larger treatment ES estimates in trials that had inadequate/unknown sequence generation (difference in ES = 0.13; 95% CI: 0.01 to 0.25) and inadequate/unknown allocation concealment (difference in ES = 0.15; 95% CI: 0.02 to 0.27). In contrast, baseline imbalance (difference in ES = 0.01, 95% CI: -0.09 to 0.12) was not associated with inflated or underestimated ES. In conclusion, treatment ES estimates were 0.13 and 0.15 larger in trials with inadequate/unknown sequence generation and inadequate/unknown allocation concealment, respectively. Therefore, authors of systematic reviews using oral health randomized controlled trials should perform sensitivity analyses based on the adequacy of sequence generation and allocation concealment.

Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955–2013

“…Bias is a threat to the quality of trials [ 38 , 39 ]; it may impact the reported treatment effect estimates in randomized clinical trials [ 40 – 44 ]. The degree of bias in randomized clinical trials of oral health intervention has decreased over time according to our study.…”

“…Because journals have a word limit that may restrict authors in reporting detailed methods, all of the methodological characteristics used might not be reported [ 52 ], thus restricting the accuracy of quality assessment tools. In the field of dentistry, treatment effect size estimates were found to be larger in trials with inadequate trial’s design characteristics which can can affect overestimation of beneficial or underestimation of harmful treatment effects within trials [ 15 , 44 ].…”

Patient morbidity after impacted third molar extraction with different flap designs. A systematic review and meta-analysis

“…Also, the screening process and summary of judgments to reach the risk of bias assessment are described transparently in the supplementary material. Nevertheless, some limitations also should be mentioned, such as the methodological issues found in the included studies, and which are characteristic of clinical trials in dentistry (21,22). The statistical heterogeneity observed was a consequence of both clinical and methodological heterogeneity among the included studies.…”

“…In addition, mention must be made of the impossibility of including more studies because they were not RCTs or did not assess pain perception based on a VAS. On the other hand, the VAS is a subjective scale, and there is evidence that it could be influenced by a lack of participants and by personnel blinding (21).…”

Odontología y Ética: sobreoferta y sobreterapia

“…La evidencia en salud es bastante inconstante y quizás algo confusa (Niederman, 1998;Saltaji et al, 2018). Desafortunadamente la ausencia de estudios bien diseñados y financiados ya sea por inversión privada (ideal) o por el estado (aceptando que Colombia destina a ciencia, tecnología e innovación menos del 0,5 % del PIB sin contar con las patéticas noticias de corrupción en este reglón del gobierno), son la realidad.…”