Estimands in cluster-randomized trials: choosing analyses that answer the right question

Abstract

Background Cluster-randomized trials (CRTs) involve randomizing groups of individuals (e.g. hospitals, schools or villages) to different interventions. Various approaches exist for analysing CRTs but there has been little discussion around the treatment effects (estimands) targeted by each. Methods We describe the different estimands that can be addressed through CRTs and demonstrate how choices between different analytic approaches can impact the interpretation of results by fundamentally changing the question being asked, or, equivalently, the target estimand. Results CRTs can address either the participant-average treatment effect (the average treatment effect across participants) or the cluster-average treatment effect (the average treatment effect across clusters). These two estimands can differ when participant outcomes or the treatment effect depends on the cluster size (referred to as ‘informative cluster size’), which can occur for reasons such as differences in staffing levels or types of participants between small and large clusters. Furthermore, common estimators, such as mixed-effects models or generalized estimating equations with an exchangeable working correlation structure, can produce biased estimates for both the participant-average and cluster-average treatment effects when cluster size is informative. We describe alternative estimators (independence estimating equations and cluster-level analyses) that are unbiased for CRTs even when informative cluster size is present. Conclusion We conclude that careful specification of the estimand at the outset can ensure that the study question being addressed is clear and relevant, and, in turn, that the selected estimator provides an unbiased estimate of the desired quantity.

Randomization, design and analysis for interdependency in aging research: no person or mouse is an island

“…Even when clustering is carefully considered, individuals in the same cluster may interfere with each other, such that the estimated (direct) effect may be biased (we use the word 'bias' several times; whether a procedure is biased depends in part on the estimand 77 ). 104 .…”

On the 12th Day of Christmas, a Statistician Sent to Me . . .

“…Thus, if, in the data analysis, a submitted article ignores obvious clustering that needs to be captured or considered, the statistical editors will ask for justification of this or for a reanalysis accounting for clustering using an approach suitable for the estimand of interest (see our stocking filler for the first day of Christmas). 52 53 54 A multilevel or mixed effects model might be recommended, for example, as this allows cluster specific baseline risks to be accounted for and enables between cluster heterogeneity in the effect of interest to be examined.…”

Commentary: Estimands in cluster trials: thinking carefully about the target of inferenceand the consequences for analysis choice

“… 3 The paper by Kahan and colleagues advises us that we also need to choose an a priori unit of inference and this choice is critical in selecting both the unit and the method of analysis. 4 We believe that the need to consider the target of inference before specifying the method of analysis has not received adequate attention in the cluster trials literature to date.…”

“…It is also important to realize that choosing the desired unit of inference is distinct from choosing the unit of analysis: regardless of whether the effect on a typical individual or typical cluster is of interest, it is possible to conduct either an individual-level analysis or a cluster-level analysis. 4 However, exactly how to carry out these analyses to ensure they answer the question of interest requires careful consideration. In the case of an individual-level analysis, two commonly used methods are the generalized linear mixed model (GLMM) or generalized estimating equations (GEEs).…”

“… 3 , 9 Whilst these individual-level methods of analysis have advantages, both GLMM and GEE models can produce biased estimates when cluster sizes are informative. 4 , 7 Conversely, a cluster-level analysis can yield an unbiased estimate for the effect on the typical cluster even when cluster sizes are informative; however, if we choose a weighting method to preserve statistical efficiency, we can inadvertently introduce a bias. 4 …”