The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials

Serres, Gaston De; Skowronski, Danuta M.; Wu, Xiaojie; Ambrose, Christopher S.

doi:10.2807/1560-7917.es2013.18.37.20585

Cited by 203 publications

(186 citation statements)

References 30 publications

Supporting

Mentioning

182

Contrasting

Order By: Relevance

“…• Measured and compared VE of pandemic and seasonal vaccines in various seasons documenting variation across seasons and sometimes across vaccines within a season in studies where classical controls were not employed (45,72,73) • In a few studies where different control groups were compared, VE estimates were sometimes similar for the test-negative design (74) and sometimes different (75); a comparison of perprotocol results from RCTs versus estimates from test-negative observational studies of an influenza vaccine and of an RSV monoclonal antibody showed close concordance (26) .…”

Section: Discussionmentioning

confidence: 97%

“…As a corollary, the number of potential biases identified for a particular design is not an indicator of the design's desirability, as these biases may be of trivial magnitude in particular cases. Some work has been done, but more work is needed, to estimate the likely magnitude of these biases (26,27,41,(61)(62)(63). Such work is important because of the high reliance on such studies to provide of the evidence base used to set vaccine policy.. Failure to account for such biases has led to large errors in the estimates of vaccine effects in observational studies, and that these errors may be amplified as they gain authority and apparent precision through the process of systematic review and meta-analysis.…”

Section: Discussionmentioning

confidence: 99%

“…While "cases" in such studies are individuals with a defined clinical syndrome who test positive for the pathogen for which the vaccine is designed, "test-negative" controls are those with the same clinical syndrome who had tested negative (1). This novel approach goes by different names and is sometimes seen as a variant of the case-control approach (26). Because it differs from a case-control study in that participants are ascertained prior to knowledge of their outcome, without a fixed ratio of outcomes in the study (27), it could be seen as a type of cohort study; indeed, arguably the first test-negative vaccine study was called an "indirect cohort" study (28).…”

Section: Test-negative Designsmentioning

confidence: 99%

“…The "cases" and "test-negative controls" both sought care for an influenza-like illness, typically defined as fever plus respiratory symptoms (17,26,(29)(30)(31), and the test-negative controls are those who test negative on the real-time reverse-transcriptase polymerase chain reaction (rRT-PCR) test for influenza virus.…”

Section: Test-negative Designsmentioning

confidence: 99%

See 3 more Smart Citations

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

2016

View full text Add to dashboard Cite

Although randomized placebo-controlled trials (RCT) are critical to establish efficacy of vaccines at the time of licensure, important remaining questions about vaccine effectiveness (VE) -used here to include individual-level measures and population-wide impact of vaccine programs --can only be answered once the vaccine is in use, from observational studies. However, such studies are inherently at risk for bias. Using a causal framework and illustrating with examples we review newer approaches to detecting and avoiding confounding and selection bias in three major classes of observational study design: cohort, case-control, and ecological studies. Studies of influenza VE, especially in seniors, are an excellent demonstration of the challenges of detecting and reducing such bias, and so we use influenza VE as a running example. We take a fresh look at the time-trend studies often dismissed as "ecological." Such designs are the only observational study design that can measure the overall effect of a vaccination program (indirect (herd) as well as direct effects), and are in fact already an important part of the evidence base for several vaccines currently in use. Despite the great strides towards more robust observational study designs, challenges lie ahead for evaluating best practices for achieving robust unbiased results from observational studies. This is critical for evaluation of national and global vaccine program effectiveness.

show abstract

Section: Discussionmentioning

confidence: 97%

Section: Discussionmentioning

confidence: 99%

Section: Test-negative Designsmentioning

confidence: 99%

Section: Test-negative Designsmentioning

confidence: 99%

See 2 more Smart Citations

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

2016

View full text Add to dashboard Cite

show abstract

“…Accuracy and validity of the TND design has been demonstrated, 38 although inherent bias and confounding factors still exist in this observational study design. In practice, small sample sizes leading to wide confidence intervals and lack of statistical power, particularly upon sample stratification, contribute to imprecise VE estimates.…”

Section: (I)variability Between Observational Studies Used To Calculamentioning

confidence: 99%

Urgent challenges in implementing live attenuated influenza vaccine

Singanayagam

Zambon

Lalvani

et al. 2018

The Lancet Infectious Diseases

View full text Add to dashboard Cite

SummaryConflicting reports have emerged about the effectiveness of the live attenuated influenza vaccine (LAIV). LAIV appears to be protecting particularly poorly against currently circulating H1N1 viruses that are derived from the 2009 pandemic H1N1 (pH1N1) viruses. During the 2015/16 influenza season, when pH1N1 was the predominant virus, studies from the United States (US) reported a complete lack of effectiveness of LAIV in children. This led to a critical decision in the US to recommend that LAIV not be used in 2016/17 and to switch to the inactivated influenza vaccine. Other countries, including the UK, Canada and Finland, have continued to recommend use of LAIV. This policy divergence and uncertainty has far reaching implications for the entire global community, given the importance of LAIV production capability for pandemic preparedness. In this Personal View, we discuss possible explanations for the observed reduced effectiveness of LAIV and highlight underpinning scientific questions. Further research to understand the reasons for these observations is essential to enable informed public health policy and commercial decisions about vaccine production and development in coming years. 3 IntroductionInfluenza vaccination remains the main strategy to control the burden of seasonal influenza disease that affects 5-10% of adults and 20-30% of children, resulting in 250,000-500,000 deaths worldwide each year. 1,2 In the US, 140 million doses of vaccine are distributed each year and vaccination is estimated to prevent nearly a quarter of predicted influenza-associated deaths. 3 Vaccination is also the primary public health response to a global influenza pandemic.Influenza vaccines contain a mixture of three or four components designed to protect against the different viruses that circulate contemporaneously as seasonal influenza viruses. Currently, these are two influenza A viruses (H1N1 and H3N2 subtypes) and two influenza B viruses (Victoria and Yamagata lineages). Vaccine components are reviewed and updated regularly by the WHO in line with observed antigenic drift of influenza viruses.The traditional inactivated influenza vaccine (IIV) was introduced in the 1940s. IIV is administered by intramuscular/intradermal injection, is licensed for use in all ages and has a good safety record. However, effectiveness rates vary, averaging around 50-60%. 4,5 More recently, live attenuated influenza vaccine (LAIV) was demonstrated to have greater efficacy than IIV in children, with absolute efficacy rates of 75-80%. [6][7][8] Internal genes of LAIV viruses are derived from a cold-adapted, attenuated strain of virus, either Ann Arbor/1960 (from the US) or Leningrad/1957 (from Russia). Haemagglutinin (HA) and neuraminidase (NA) surface antigens are engineered to be representative of currently circulating strains. LAIV is nasally administered and vaccine viruses replicate only in the air-cooled environment of the human upper respiratory tract. This results in a mild and self-limiting infection; virus cannot replicate at t...

show abstract

Effectiveness Associated With BNT162b2 Vaccine Against Emergency Department and Urgent Care Encounters for Delta and Omicron SARS-CoV-2 Infection Among Adolescents Aged 12 to 17 Years

et al. 2022

View full text Add to dashboard Cite

IMPORTANCEData about the duration of protection of 2 and 3 doses of BNT162b2 in children and adolescents are needed to help inform recommendations for boosters in this age group. OBJECTIVE To evaluate vaccine effectiveness (VE) and durability associated with 2 doses of BNT162b2 against Delta-and Omicron-related emergency department (ED) and urgent care (UC) encounters among adolescents aged 12 to 17 years and to estimate VE associated with 3 doses against these same outcomes. DESIGN, SETTING, AND PARTICIPANTS This test-negative case-control study was conducted at Kaiser Permanente Southern California, an integrated health care system using electronic health records in the US. Participants included Kaiser Permanente Southern California members ages 12 to 17 years with an ED or UC encounter from November 1, 2021, through March 18, 2022, for acute respiratory infection who were tested for SARS-CoV-2 via a reverse transction-polymerase chain reaction test. Analyses were conducted from March 21 to June 22, 2022. EXPOSURES BNT162b2 vaccination status ascertained from electronic health records and state registry data. MAIN OUTCOMES AND MEASURES The main outcome was VE associated with BNT162b2 against ED and UC encounters related to Delta or Omicron variant SARS-CoV-2 infection. RESULTSAnalyses were conducted among 3168 adolescents, including 1004 with ED visits and 2164 with UC visits. Median (IQR) age was 15 (13-16) years, and 1461 (46.1%) were boys. In adjusted analyses, VE associated with 2 doses of BNT162b2 against ED or UC encounters was highest within the first 2 months for both Delta (89% [95% CI, 69% to 96%]) and Omicron (73% [95% CI, 54% to 84%]) variants but waned to 49% (95% CI, 27% to 65%) for the Delta variant and 16% (95% CI, −7% to 34%) for the Omicron variant at 6 months and beyond. A third dose of BNT162b2 was associated with improved protection against the Omicron variant (87% [95% CI, 72% to 94%]) after a median (IQR) of 19 (9-32) days after dose 3. CONCLUSIONS AND RELEVANCEThese findings suggest that 2 doses of the BNT162b2 COVID-19 vaccine were associated with high levels of protection against ED and UC encounters related to the Delta and Omicron variants of SARS-CoV-2 in the first few months after vaccination. However, effectiveness waned over time, especially against Omicron. A third dose of BNT162b2 was associated with improved protection against Omicron beyond that seen initially after 2 doses, underscoring the importance of boosters for adolescents aged 12 to 17 years.

show abstract

The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials

Abstract: Binary file ES_Abstracts_Final_ECDC.txt matches

Cited by 203 publications

References 30 publications

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

Observational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies

Urgent challenges in implementing live attenuated influenza vaccine

Effectiveness Associated With BNT162b2 Vaccine Against Emergency Department and Urgent Care Encounters for Delta and Omicron SARS-CoV-2 Infection Among Adolescents Aged 12 to 17 Years

Contact Info

Product

Resources

About