Testing behaviour may bias observational studies of vaccine effectiveness

Abstract

Background Recent observational studies have suggested that vaccines for the omicron variant of SARS-Cov2 may have little or no effect in preventing infection. However, the observed effects may be confounded by patient factors and preventive behaviours or vaccine-related differences in testing behaviour. To assess the potential degree of confounding, we aimed to estimate differences in testing behaviour between unvaccinated and vaccinated populations. Methods We recruited 1,526 Australian adults for an online randomised study about COVID testing between October and November 2021, and collected self-reported vaccination status and three measures of COVID-19 testing behaviour. We examined the association between testing intentions and vaccination status in the cross-sectional baseline data of this trial. Results Of the 1,526 participants (mean age 31 years): 22% had a COVID-19 test in the past month and 61% ever; 17% were unvaccinated, 11% were partially vaccinated (1 dose), 71% were fully vaccinated (2+ doses). Fully vaccinated participants were twice as likely to report positive COVID testing intentions than those who were unvaccinated (p<.001). Partially vaccinated participants had less positive intentions than those fully vaccinated (p<.001) but higher intentions than those who were unvaccinated (p=.002). Discussion For all three measures vaccination predicted greater COVID testing intentions. If the unvaccinated tested at half the rate of the vaccinated, a true vaccine effectiveness of 30% could appear to be a negative observed vaccine effectiveness of -40%. Assessing vaccine effectiveness should use methods to account for differential testing behaviours. Test negative designs are currently the preferred option, but its assumptions should be more thoroughly examined.

Waning effectiveness of BNT162b2 and ChAdOx1 covid-19 vaccines over six months since second dose: OpenSAFELY cohort study using linked electronic health records

“…This limitation should not affect results for the ≥65 subgroup, most of whom are retired, or comparisons between BNT162b2 and ChAdOx1. Thirdly, consistent with an Australian survey,33 we found that unvaccinated people had tested less frequently than vaccinated people during the pre-vaccine rollout period when widespread testing was available (table 1) and were considerably less likely to be tested during follow-up (supplementary table S7). Fourthly, differential depletion of susceptible people in the unvaccinated groups over time may lead to attenuation of hazard ratios even when true vaccine effectiveness does not change.…”

Effectiveness of COVID-19 vaccines against Omicron or Delta symptomatic infection and severe outcomes

“…If asymptomatic vaccinated individuals are more likely to be tested (e.g., healthcare workers, individuals exposed to cases), vaccinated Omicron cases would be more likely to be identified compared to unvaccinated Omicron cases, leading to an underestimate of VE. 50 Restricting the analysis to symptomatic individuals, as presented here, should mitigate this potential source of bias.…”

“…However, if vaccinated individuals are more likely to be tested and therefore captured in our data than unvaccinated individuals, then VE will be underestimated. 50 Fourth, symptom information is not available for all laboratories submitting to Ontario's centralized system; as such, symptomatic individuals who were tested but without this information recorded would have been excluded in our study. 23 Fifth, despite ongoing high case counts, we were unable to extend our time period past the end of December due to restricted test eligibility and access, reduced SGTF screening and the likelihood of misclassification due to the potential rise of the BA.2 sub-lineage of Omicron, as noted elsewhere, 38 51 but so far has been minimal in Ontario.…”

Waning effectiveness of BNT162b2 and ChAdOx1 COVID-19 vaccines over six months since second dose: a cohort study using linked electronic health records

“…The large study size and large numbers of outcome events led to precise estimates of vaccine effectiveness according to vaccine brand and time since second vaccine dose. We accounted for risk-dependent vaccine allocation by separating the cohort into subgroups based on JCVI group, 26 and conducting analyses within strata defined by JCVI group, eligibility date for primary vaccination and geographical region. Our analyses also excluded individuals with a pre-vaccine rollout record of SARS-CoV-2 infection and accounted for rapid changes in COVID-19 incidence with calendar time, censoring due to occurrence of outcome events, and attenuation of comparison groups because receipt of receipt of first vaccine dose by unvaccinated individuals and third dose by fully vaccinated individuals.…”