The Use of Test-negative Controls to Monitor Vaccine Effectiveness

Chua, Huiying; Feng, Luzhao; Lewnard, Joseph A; Sullivan, Sheena G.; Blyth, Christopher C; Lipsitch, Marc; Cowling, Benjamin J.

doi:10.1097/ede.0000000000001116

Cited by 126 publications

(142 citation statements)

References 379 publications

(43 reference statements)

Supporting

Mentioning

129

Contrasting

Unclassified

Order By: Relevance

“…through matching of controls to cases in key characteristics, such as sociodemographic characteristics, or other risk factors including occupational exposure or geographic location. A "test-negative" design, where laboratory confirmed NiV patients are considered as cases and suspected NiV patients that tested negative are considered as controls, could help preventing some biases related to healthcare seeking behavior [21][22][23]. Evaluating vaccine efficacy based on an alternative outcome that cannot be causally affected by the vaccine can help testing for potential biases in the study [21].…”

Section: Discussionmentioning

confidence: 99%

Assessing the feasibility of Nipah vaccine efficacy trials based on previous outbreaks in Bangladesh

Nikolay

Lipsitch

Rahman

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

BackgroundNipah virus (NiV) is an emerging, bat-borne pathogen that can be transmitted from person-to-person. Vaccines are currently being developed for NiV, and studies funded to evaluate their safety and immunogenicity, so that they could possibly be used to contain outbreaks. An important unanswered question is whether it will be possible to evaluate the efficacy of vaccine candidates in phase III clinical trials in a context where spillovers from the zoonotic reservoir are infrequent and associated with small outbreaks. The objective of this study was to investigate the feasibility of conducting a phase III vaccine trial in Bangladesh, the only country reporting regularly NiV cases.MethodsWe used simulations based on previously observed NiV cases from Bangladesh, an assumed vaccine efficacy of 90% and other NiV vaccine target characteristics, to compare three vaccination study designs: (i) cluster randomized ring vaccination, (ii) cluster randomized mass vaccination, and (iii) an observational case-control study design.ResultsThe simulations showed that, assuming a ramp-up period of 10 days and a mean hospitalization delay of 4 days, it would take 516 years and over 163,000 vaccine doses to run a ring vaccination trial under current epidemic conditions. A cluster-randomized trial in the two most affected districts would take 81 years and 2.3 million vaccine doses. An observational case-control design in these two districts would require seven years and 2.5 million vaccine doses.DiscussionWithout a change in the epidemiology of NiV, ring vaccination or cluster-randomized trials are unlikely to be completed within a reasonable time window. In this light, the remaining options are: (i) not conducting a phase III trial until the epidemiology of NiV changes, (ii) identifying alternative ways to licensure such as observational studies or controlled studies in animals such as in the US Food and Drug Administration’s (FDA) Animal Rule.

show abstract

Section: Discussionmentioning

confidence: 99%

Assessing the feasibility of Nipah vaccine efficacy trials based on previous outbreaks in Bangladesh

Nikolay

Lipsitch

Rahman

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The TND is an alternative to the TCC that has become the default approach for studies of vaccines against influenza, rotavirus, and other pathogens. 13 In the TND, both cases and controls are selected from among individuals who receive diagnostic tests for a pathogen of interest; conventionally, individuals are tested because they experience a clinical syndrome which may be caused by the vaccine-targeted pathogen or other agents. Cases are those testing positive, and controls are those testing negative, placing a heavy reliance on diagnostic accuracy.…”

Section: Overview Of the Designsmentioning

confidence: 99%

Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines

Lewnard

Patel

Jewell

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Observational studies of the effectiveness of vaccines to prevent COVID-19 are needed to inform real-world use. These are now in planning amid the ongoing rollout of SARS-CoV-2 vaccines globally. While traditional case-control (TCC) and test-negative design (TND) studies feature prominently among strategies used to assess vaccine effectiveness, such studies may encounter important threats to validity. Here we review the theoretical basis for estimation of vaccine direct effects under TCC and TND frameworks, addressing specific natural history parameters of SARS-CoV-2 infection and COVID-19 relevant to these designs. Bias may be introduced by misclassification of cases and controls, particularly when clinical case criteria include common, non-specific indicators of COVID-19. When using diagnostic assays with high analytical sensitivity for SARS-CoV-2 detection, individuals testing positive may be counted as cases even if their symptoms are due to other causes. The TCC may be particularly prone to confounding due to associations of vaccination with healthcare-seeking behavior or risk of infection. The TND reduces but may not eliminate this confounding, for instance if individuals who receive vaccination seek care or testing for less-severe infection. These circumstances indicate the two study designs cannot be applied naively to datasets gathered through public health surveillance or administrative sources. We suggest practical strategies to reduce bias in vaccine effectiveness estimates at the study design and analysis stages.

show abstract

“…A recent review of TNDs to mosquito vaccine effectiveness discusses 348 such studies. 13 In a TND, test-positives play the role of our case counts in Section 2 and are ascertained through attendance, diagnosis, and testing at a clinic or other health-care setting. Subsequently, a critical component of the TND is the definition of test-negatives.…”

Section: The Test-negative Designmentioning

confidence: 99%

“…The design and analytical methods were recently extended to cluster randomized interventions, yielding the so‐called cluster randomized test‐negative design (CR‐TND). A recent review of TNDs to mosquito vaccine effectiveness discusses 348 such studies …”

Section: Differential Case Ascertainmentmentioning

confidence: 99%

Analysis of counts for cluster randomized trials: Negative controls and test‐negative designs

Dufault

Jewell

2020

Statistics in Medicine

View full text Add to dashboard Cite

In cluster randomized trials (CRTs), the outcome of interest is often a count at the cluster level. This occurs, for example, in evaluating an intervention with the outcome being the number of infections of a disease such as HIV or dengue or the number of hospitalizations in the cluster. Standard practice analyzes these counts through cluster outcome rates using an appropriate denominator (eg, population size). However, such denominators are sometimes unknown, particularly when the counts depend on a passive community surveillance system. We consider direct comparison of the counts without knowledge of denominators, relying on randomization to balance denominators. We also focus on permutation tests to allow for small numbers of randomized clusters. However, such approaches are subject to bias when there is differential ascertainment of counts across arms, a situation that may occur in CRTs that cannot implement blinded interventions. We suggest the use of negative control counts as a method to remove, or reduce, this bias, discussing the key properties necessary for an effective negative control. A current example of such a design is the recent extension of test‐negative designs to CRTs testing community‐level interventions. Via simulation, we compare the performance of new and standard estimators based on CRTs with negative controls to approaches that only use the original counts. When there is no differential ascertainment by intervention arm, the count‐only approaches perform comparably to those using debiasing negative controls. However, under even modest differential ascertainment, the count‐only estimators are no longer reliable.

show abstract

The Use of Test-negative Controls to Monitor Vaccine Effectiveness

Cited by 126 publications

References 379 publications

Assessing the feasibility of Nipah vaccine efficacy trials based on previous outbreaks in Bangladesh

Assessing the feasibility of Nipah vaccine efficacy trials based on previous outbreaks in Bangladesh

Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines

Analysis of counts for cluster randomized trials: Negative controls and test‐negative designs

Contact Info

Product

Resources

About