Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation

Abstract

BackgroundRandomised controlled trials (RCT) are highly influential upon medical decisions. Thus RCTs must not distort the truth. One threat to internal trial validity is the correct prediction of future allocations (selection bias). The Berger-Exner test detects such bias but has not been widely utilized in practice. One reason for this non-utilisation may be a lack of information regarding its test accuracy. The objective of this study is to assess the accuracy of the Berger-Exner test on the basis of relevant simulations for RCTs with dichotomous outcomes.MethodsSimulated RCTs with various parameter settings were generated, using R software, and subjected to bias-free and selection bias scenarios. The effect size inflation due to bias was quantified. The test was applied in both scenarios and the pooled sensitivity and specificity, with 95% confidence intervals for alpha levels of 1%, 5%, and 20%, were computed. Summary ROC curves were generated and the relationships of parameters with test accuracy were explored.ResultsAn effect size inflation of 71% - 99% was established. Test sensitivity was 1.00 (95% CI: 0.99 – 1.00) for alpha level 1%, 5%, and 20%; test specificity was 0.94 (95% CI: 0.93 – 0.96); 0.82 (95% CI: 0.80 – 0.84), and 0.56 (95% CI: 0.54 – 0.58) for alpha 1%, 5%, and 20%, respectively. Test accuracy was best with the maximal procedure used with a maximum tolerated imbalance (MTI) = 2 as the randomisation method at alpha 1%.ConclusionsThe results of this simulation study suggest that the Berger-Exner test is generally accurate for identifying third-order selection bias.Electronic supplementary materialThe online version of this article (doi:10.1186/1471-2288-14-114) contains supplementary material, which is available to authorized users.

Caries-Preventive Effect of High-Viscosity Glass Ionomer and Resin-Based Fissure Sealants on Permanent Teeth: A Systematic Review of Clinical Trials

“…Such trials should apply adequate methods for randomised subject allocation and allocation concealment. In addition, future trials should include the Berger-Exner test [ 41 ] in order to investigate any possible inclusion of third order selection bias during the trial.…”

A Simulation Study of Threats to Validity in Quasi-Experimental Designs: Interrelationship between Design, Measurement, and Analysis

“…A similar methodology has been already found as useful to study the influence of threats to validity in other fields; e.g., (a) in forest research, Ficko and Boncina (2014) operationalized the influence of response style bias and the robustness of statistical methods in the results using simulations and including latent variables in the models representing those threats to validity; and (b) in medical research, Mickenautsch et al (2014) studied the inflation of effect size owing to selection bias using simulations. In the current study, we show the application and usefulness of simulations and the SEM framework in social sciences, specifically in psychology, to detect the influence of other different threats to validity.…”

A randomized controlled trial of high volume simethicone to improve visualization during capsule endoscopy

“…The randomization schema was concealed electronically by REDCap and the randomization allocation was available one subject at a time and only after completion of study enrollment. There were no concerns for the integrity of allocation concealment, which was also supported by the Berger-Exner test (p = 0.29 (treatment), and p = 0.28 (control)) [ 21 , 22 ].…”