Clarifying selection bias in cluster randomized trials

Abstract

Background In cluster randomized trials, patients are typically recruited after clusters are randomized, and the recruiters and patients may not be blinded to the assignment. This often leads to differential recruitment and consequently systematic differences in baseline characteristics of the recruited patients between intervention and control arms, inducing post-randomization selection bias. We aim to rigorously define causal estimands in the presence of selection bias. We elucidate the conditions under which standard covariate adjustment methods can validly estimate these estimands. We further discuss the additional data and assumptions necessary for estimating causal effects when such conditions are not met. Methods Adopting the principal stratification framework in causal inference, we clarify there are two average treatment effect (ATE) estimands in cluster randomized trials: one for the overall population and one for the recruited population. We derive analytical formula of the two estimands in terms of principal-stratum-specific causal effects. Furthermore, using simulation studies, we assess the empirical performance of the multivariable regression adjustment method under different data generating processes leading to selection bias. Results When treatment effects are heterogeneous across principal strata, the average treatment effect on the overall population generally differs from the average treatment effect on the recruited population. A naïve intention-to-treat analysis of the recruited sample leads to biased estimates of both average treatment effects. In the presence of post-randomization selection and without additional data on the non-recruited subjects, the average treatment effect on the recruited population is estimable only when the treatment effects are homogeneous between principal strata, and the average treatment effect on the overall population is generally not estimable. The extent to which covariate adjustment can remove selection bias depends on the degree of effect heterogeneity across principal strata. Conclusion There is a need and opportunity to improve the analysis of cluster randomized trials that are subject to post-randomization selection bias. For studies prone to selection bias, it is important to explicitly specify the target population that the causal estimands are defined on and adopt design and estimation strategies accordingly. To draw valid inferences about treatment effects, investigators should (1) assess the possibility of heterogeneous treatment effects, and (2) consider collecting data on covariates that are predictive of the recruitment process, and on the non-recruited population from external sources such as electronic health records.

Leveraging baseline covariates to analyze small cluster-randomized trials with a rare binary outcome

“…Fourth, we have primarily focused on the role of covariate adjustment for reducing chance imbalance and improving precision in CRTs and have not considered the use of covariates to address post-randomization selection nor recruitment bias, which involve yet another prevailing complexity in CRTs. Previous studies suggest that covariate adjustment helps reduce selection bias to a certain extent (Leyrat et al, 2013(Leyrat et al, , 2014 but is not a definitive solution especially in the presence of unmeasured variables predictive of recruitment (Li et al, 2021). The baseline imbalance we observed in our application to RESTORE trial (Table 3), therefore, might also be due to selection bias if the identification or recruitment of subjects was unblinded.…”

“…With regulators such as the U.S. Food and Drug Administration and the European Medicines Agency recommending prognostic baseline covariate adjustment in individually-randomized trials, there remains sufficient interest in fully understanding whether covariate adjustment can improve efficiency and precision in CRTs, especially when the number of clusters is often limited. Third, covariate adjustment may reduce selection or recruitment bias compared to the unadjusted analysis in CRTs (Leyrat et al, 2013(Leyrat et al, , 2014Li et al, 2021).…”

Randomization, design and analysis for interdependency in aging research: no person or mouse is an island

“…Ignoring clustering and nesting during analyses can lead to an inflated type I error rate 3,[69][70][71][72] . There are additional issues, such as census recruitment or enrolling via cluster random sampling, a twostage process in which the population is divided into clusters and a subset of the clusters is randomly selected, as opposed to investigator led selection of clusters, which can be argued to induce bias and we refer the reader elsewhere for detailed discussions [73][74][75] .…”

Defining and estimating effects in cluster randomized trials: A methods comparison

“…Likewise, using real data from a CRT, Balzer et al recently showed the adjusted analysis was five times more efficient than an unadjusted analysis for the same parameter 16 . While our focus is on the potential of covariate adjustment to improve precision, we note that in other settings covariate adjustment may be essential to reducing bias due to missingness, selection, or restricted randomization 16‐25 …”

“…The $$$ {N}_j $$$ study participants could be randomly sampled within a cluster or be a census of all persons in a cluster. Discussion of more complex settings with systematic sampling or other forces of selection bias is important but beyond the scope of this paper 16,20‐23 . Here, the study participants are representative of or comprise the cluster.…”