Understanding COVID-19 Vaccine Effectiveness against Death Using a Novel Measure: COVID Excess Mortality Percentage

Abstract

COVID-19 vaccines have saved millions of lives; however, understanding the long-term effectiveness of these vaccines is imperative to developing recommendations for booster doses and other precautions. Comparisons of mortality rates between more and less vaccinated groups may be misleading due to selection bias, as these groups may differ in underlying health status. We studied all adult deaths during the period of 1 April 2021–30 June 2022 in Milwaukee County, Wisconsin, linked to vaccination records, and we used mortality from other natural causes to proxy for underlying health. We report relative COVID-19 mortality risk (RMR) for those vaccinated with two and three doses versus the unvaccinated, using a novel outcome measure that controls for selection effects. This measure, COVID Excess Mortality Percentage (CEMP), uses the non-COVID natural mortality rate (Non-COVID-NMR) as a measure of population risk of COVID mortality without vaccination. We validate this measure during the pre-vaccine period (Pearson correlation coefficient = 0.97) and demonstrate that selection effects are large, with non-COVID-NMRs for two-dose vaccinees often less than half those for the unvaccinated, and non-COVID NMRs often still lower for three-dose (booster) recipients. Progressive waning of two-dose effectiveness is observed, with an RMR of 10.6% for two-dose vaccinees aged 60+ versus the unvaccinated during April–June 2021, rising steadily to 36.2% during the Omicron period (January–June, 2022). A booster dose reduced RMR to 9.5% and 10.8% for ages 60+ during the two periods when boosters were available (October–December, 2021; January–June, 2022). Boosters thus provide important additional protection against mortality.

Selection Effects and COVID-19 Mortality Risk after Pfizer vs. Moderna Vaccination: Evidence from Linked Mortality and Vaccination Records

“…Among other studies of VE against death, some study only a single vaccine type (e.g., Israeli studies of Pfizer; manufacturer-sponsored studies). Of those that study both vaccines, many report only combined results rather than vaccine-specific results [ 5 , 10 , 11 , 12 , 13 , 14 , 30 ]. Some do not report separate results for homologous versus heterologous vaccination.…”

“…Failure to address these selection effects can lead to biased estimates of vaccine effectiveness (VE), relative mortality risk (RMR) for the vaccinated relative to the unvaccinated, and RMR for Pfizer versus Moderna vaccinees [ 4 ]. In prior work, we provide evidence for strong selection effects on which persons are vaccinated; two-dose vaccinees are healthier than the unvaccinated, and three-dose vaccinees are generally healthier than two-dose vaccinees [ 5 ]. Here, we extend that work and provide evidence for important selection effects in which vaccine people receive.…”

“…Many studies have reported real-world evidence on vaccine effectiveness (VE) for the mRNA vaccines against infection, hospitalization, and death (for brevity, we cite principally systematic reviews) [ 6 , 7 , 8 , 9 ]. However, many studies report results for both vaccines together [ 5 , 10 , 11 , 12 , 13 , 14 , 15 ]. Most studies that report vaccine-specific results have limited controls for individual characteristics, often only age and gender; use research designs that are prone to selection effects, often a test-negative design, which compares people with positive COVID-19 tests to controls who test negative; and/or cover limited time periods or populations (e.g., US veterans).…”

“…In prior work, we propose, validate, and use a novel approach for addressing selection when studying COVID-19 mortality risk [ 4 , 5 , 21 ]. We use the natural mortality rate from causes other than COVID-19 (non-COVID-19 NMR) as a proxy for unobserved health and thus background mortality risk, within population groups defined by age, gender, vaccination status, and other characteristics.…”

Evidence on COVID-19 Mortality and Disparities Using a Novel Measure, COVID excess mortality percentage: Evidence from Indiana, Wisconsin, and Illinois

“…We can validate the CEMP measure by examining the association between natural mortality rates and COVID-19 mortality rates during 2020, when vaccines were not yet available ( Fig 1 , Panel A). The Pearson correlation coefficient between Non-Covid-NMR in April-December 2019 and COVID-19 MR during early and late stages of the pandemic, within cells defined by age group, gender, race/ethnicity and state, ranges is 0.90 ( Fig 1 , Panel A) [ 15 ]. The high correlation is not surprising, because many specific risk factors for COVID-19 mortality (e.g., age, gender, obesity, diabetes) are also correlated with non-COVID mortality [ 13 ].…”