Forecasts of the trajectory of an infectious agent can help guide public health decision making. A traditional approach to forecasting fits a computational model to structured data and generates a predictive distribution. However, human judgment has access to the same data as computational models plus experience, intuition, and subjective data. We propose a chimeric ensemble-a combination of computational and human judgment forecasts-as a novel approach to predicting the trajectory of an infectious agent. Each month from January, 2021 to June, 2021 we asked two generalist crowds, using the same criteria as the COVID-19 Forecast Hub, to submit a predictive distribution over incident cases and deaths at the US national level either two or three weeks into the future and combined these human judgment forecasts with forecasts from computational models submitted to the COVID-19 Forecasthub into a chimeric ensemble. We find a chimeric ensemble compared to an ensemble including only computational models improves predictions of incident cases and shows similar performance for predictions of incident deaths. A chimeric ensemble is a flexible, supportive public health tool and shows promising results for predictions of the spread of an infectious agent.
Background: An increase in reported human infections by the monkeypox virus (MPXV) has been observed in multiple non-endemic countries. Forecasts of transmission and disease burden associated with MPXV can support public health decision making. However, historical data that can be used to train computational forecasts is sparse. Here we show how crowdsourced human judgment can generate probabilistic predictions of the potential evolution of the international MPXV outbreak before robust computational models are prepared to provide such forecasts. Methods: We posed 8 questions associated with the monkeypox outbreak on the Metaculus forecasting platform. A total of 686 original and revised probabilistic predictions from 222 human forecasters were submitted to the forecasting platform from May 19th, 2022 to May 24, 2022. A performance based ensemble algorithm combined these individual predictions into ensemble forecasts. Findings: At time of writing, human judgment ensemble forecasts predict that the number of incident cases in the US, Canada, and Europe will continue to increase and the virus will continue to spread to multiple additional countries. Ensemble forecasts predict the World Health Organization will not declare human monkeypox a Public Health Emergency of International Concern before Dec 31, 2022.Interpretation: Human judgment forecasting is a rapid and readily adaptable approach that may improve situational awareness, synthesize available evidence, and meet public health needs as an outbreak evolves.
Forecasts of the trajectory of an infectious agent can help guide public health decision making. A traditional approach to forecasting fits a computational model to structured data and generates a predictive distribution. However, human judgment has access to the same data as computational models plus experience, intuition, and subjective data. We propose a chimeric ensemble—a combination of computational and human judgment forecasts—as a novel approach to predicting the trajectory of an infectious agent. Each month from January, 2021 to June, 2021 we asked two generalist crowds, using the same criteria as the COVID-19 Forecast Hub, to submit a predictive distribution over incident cases and deaths at the US national level either two or three weeks into the future and combined these human judgment forecasts with forecasts from computational models submitted to the COVID-19 Forecasthub into a chimeric ensemble. We find a chimeric ensemble compared to an ensemble including only computational models improves predictions of incident cases and shows similar performance for predictions of incident deaths. A chimeric ensemble is a flexible, supportive public health tool and shows promising results for predictions of the spread of an infectious agent.
Aggregated human judgment forecasts for COVID-19 targets of public health importance are accurate, often outperforming computational models. Our work shows aggregated human judgment forecasts for infectious agents are timely, accurate, and adaptable, and can be used as tool to aid public health decision making during outbreaks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.