The Shiny Balancer - software and imbalance criteria for optimally balanced treatment allocation in small RCTs and cRCTs

Abstract

BackgroundIn randomised controlled trials with only few randomisation units, treatment allocation may be challenging if balanced distributions of many covariates or baseline outcome measures are desired across all treatment groups. Both traditional approaches, stratified randomisation and allocation by minimisation, have their own limitations. A third method for achieving balance consists of randomly choosing from a preselected list of sufficiently balanced allocations. As with minimisation, this method requires that heterogeneity between treatment groups is measured by specified imbalance metrics. Although certain imbalance measures are more commonly used than others, to the author's knowledge there is no generally accepted “gold standard”, neither for categorical and even less so for continuous variables.MethodsAn intuitive and easily accessible web-based software tool was developed which allows for balancing multiple variables of different types and using various imbalance metrics. Different metrics were compared in a simulation study.ResultsUsing simulated data, it could be shown that for categorical variables, χ2-based imbalance measures seem to be viable alternatives to the established “quadratic imbalance” metric. For continuous variables, using the area between the empirical cumulative distribution functions or the largest difference in the three pairs of quartiles is recommended to measure imbalance. Another imbalance metric suggested in the literature for continuous variables, the (symmetrised) Kullback-Leibler divergence, should be used with caution.ConclusionThe Shiny Balancer offers the possibility to visually explore the balancing properties of several well established or newly suggested imbalance metrics, and its use is particularly advocated in clinical studies with few randomisation units, as it is typically the case in cluster randomised trials.Electronic supplementary materialThe online version of this article (10.1186/s12874-018-0551-5) contains supplementary material, which is available to authorized users.

The Effect of Financial Incentives on Quality Measures in the Treatment of Diabetes Mellitus: a Randomized Controlled Trial

“…The current proportion of the clinical QM, number of participating GPs per practice, GP network participation, and number of patients with diabetes mellitus were used to stratify randomization. We used the shiny balancer software for randomization 22 and generated 100 balanced allocation schemes. The settings for the shiny balancer software are displayed in Additional File 1 , Table 1 .…”

Design and analysis of three‐arm parallel cluster randomized trials with small numbers of clusters

“…A commonly used balance score for two‐arm trials was proposed by Raab and Butcher, which is a sum of standardized mean differences between the two arms, 8 but could include any statistic that compares the distance between the distributions of the covariate(s) of interest in the different trial arms. For example, Grischott 21 provides functionality for 15 different measures in an online constrained‐randomization app.…”

Improving inappropriate medication and information transfer at hospital discharge: study protocol for a cluster RCT

“…Sufficiently large blocks of clusters [28] will be allocated to the study arms using covariate-constrained randomisation [29], i.e. by randomly choosing (by an independent third party) from a set of randomly generated allocation schemes with sufficient balance in terms of hospital types (acute-care vs rehabilitation, rural vs central, academic vs non-academic) as well as type (medical discipline) and size (number of beds) of hospital wards.…”