Analysis of counts for cluster randomized trials: Negative controls and test‐negative designs

Abstract

In cluster randomized trials (CRTs), the outcome of interest is often a count at the cluster level. This occurs, for example, in evaluating an intervention with the outcome being the number of infections of a disease such as HIV or dengue or the number of hospitalizations in the cluster. Standard practice analyzes these counts through cluster outcome rates using an appropriate denominator (eg, population size). However, such denominators are sometimes unknown, particularly when the counts depend on a passive community surveillance system. We consider direct comparison of the counts without knowledge of denominators, relying on randomization to balance denominators. We also focus on permutation tests to allow for small numbers of randomized clusters. However, such approaches are subject to bias when there is differential ascertainment of counts across arms, a situation that may occur in CRTs that cannot implement blinded interventions. We suggest the use of negative control counts as a method to remove, or reduce, this bias, discussing the key properties necessary for an effective negative control. A current example of such a design is the recent extension of test‐negative designs to CRTs testing community‐level interventions. Via simulation, we compare the performance of new and standard estimators based on CRTs with negative controls to approaches that only use the original counts. When there is no differential ascertainment by intervention arm, the count‐only approaches perform comparably to those using debiasing negative controls. However, under even modest differential ascertainment, the count‐only estimators are no longer reliable.

Double Negative Control Inference in Test-Negative Design Studies of Vaccine Effectiveness

“…In such settings, both estimators β and β(X) produced by Algorithms 1 and 2 may be severely biased because Assumption 2 may no longer be valid. Crucially, we note that this form of selection bias can be present even in context of a randomized trial in which vaccination/treatment is assigned completely at random, if the outcome is ascertained using a TND, for example in the cluster-randomized test-negative design studies of community-level dengue intervention effectiveness Anders et al (2018), Dufault and Jewell (2020), Jewell et al (2019), and Wang et al (2022). In this Section, we provide sufficient conditions for identification under treatment-induced selection.…”

Randomization Inference for Cluster-Randomized Test-Negative Designs with Application to Dengue Studies: Unbiased estimation, Partial compliance, and Stepped-wedge design

“…Since Anders et al (2018) first introduced the CR-TND, Jewell et al (2019) proposed a series of cluster-level estimators for the relative risk, basing inference on permutation distributions and approximations thereof. Dufault and Jewell (2020) later addressed the impact of differential healthcare-seeking behavior caused by unblinded intervention assignment by considering a case-only analysis strategy. A further challenge of differential healthcare-seeking behavior occurs when the latter varies across clusters, which is not considered in Dufault and Jewell (2020).…”

“…Dufault and Jewell (2020) later addressed the impact of differential healthcare-seeking behavior caused by unblinded intervention assignment by considering a case-only analysis strategy. A further challenge of differential healthcare-seeking behavior occurs when the latter varies across clusters, which is not considered in Dufault and Jewell (2020). Furthermore, it is valuable to account for non-compliance or partial compliance with the assigned intervention.…”

“…We make the following assumptions for estimating λ, which we refer to as the assumptions for a CR-TND. These assumptions are also seen in Haber et al (2015); Anders et al (2018); Dufault and Jewell (2020), where they are described under the super-population framework (i.e., assuming the data from each cluster i are independent, identically distributed samples from a common distribution). We modify these assumptions to fit into the randomization inference framework as follows, which places no additional assumption on the counterfactual distributions.…”

Ratio estimators of intervention effects on event rates in cluster randomized trials

“…That is, they may determine only the number of events in a cluster, without identifying which individual members of the cluster experienced the events. Furthermore, the denominators for standard practice of calculating event rates may not be available 4 . The advantage of passive surveillance is that the monetary and opportunity cost of data collection can be much reduced.…”