Sensitivity Analysis of Excess Mortality due to the COVID‐19 Pandemic

Abstract

Estimating excess mortality is challenging. The metric depends on the expected mortality level, which can differ based on given choices, such as the method and the time series length used to estimate the baseline. However, these choices are often arbitrary, and are not subject to any sensitivity analysis. We bring to light the importance of carefully choosing the inputs and methods used to estimate excess mortality. Drawing on data from 26 countries, we investigate how sensitive excess mortality is to the choice of the mortality index, the number of years included in the reference period, the method, and the time unit of the death series. We employ two mortality indices, three reference periods, two data time units, and four methods for estimating the baseline. We show that excess mortality estimates can vary substantially when these factors are changed, and that the largest variations stem from the choice of the mortality index and the method. We also find that the magnitude of the variation in excess mortality is country‐specific, resulting in cross‐country rankings changes. Finally, based on our findings, we provide guidelines for estimating excess mortality.

Monthly excess mortality across counties in the United States during the Covid-19 pandemic, March 2020 to February 2022

“…Third, the estimates produced by this study identify more excess deaths than a prior analysis of county-level excess deaths during 2020 due to differences in methods and time horizons. (4,35) We also found a lower percentage of excess deaths assigned to Covid-19 since we used underlying cause of death data rather than assessing whether Covid-19 appeared anywhere on the death certificate. Fourth, we were unable to distinguish between excess deaths from natural causes and excess deaths indirectly related to the pandemic.…”

Estimates of excess mortality for the five Nordic countries during the Covid-19 pandemic 2020-2021

“…Yet this approach, as applied e.g. by Karlinsky and Kobak, 10 and Economist, 15 does not include any assumptions beyond linearity and thus serves an important purpose as sensitivity tests for understanding and comparing the data, which is essential 16 . Methods that use fixed functional (e.g., sinusoidal) forms to smooth out curves and e.g., reduce the impact of unusual influenza seasons or heat waves also exist, e.g., euromomo.…”

What should be the baseline when calculating excess mortality? New approaches suggest that we have underestimated the impact of the COVID-19 pandemic and previous winter peaks

“…The concept of excess mortality has received considerable attention since the start of the pandemic and is now used widely as a metric to quantify the impact of the pandemic, comparing observed deaths with those that would be expected if death rates had remained the same as in previous years ( Iacobucci, 2021 ). It compares week- or month-specific death rates over a year with some baseline mortality level, such as (for example) the five years preceding the year being studied ( Nepomuceno et al, 2022 ; Schöley, 2021 ). The magnitude of the deviation from previous years is taken as quantifying the deaths that would not have occurred if there had not been a pandemic.…”