Real-time COVID-19 forecasting: challenges and opportunities of model performance and translation

Nixon, Kristen; Jindal, Sonia; Parker, Felix; Marshall, Maximilian; Reich, Nicholas G.; Ghobadi, Kimia; Lee, Elizabeth C.; Truelove, Shaun; Gardner, Lauren

doi:10.1016/s2589-7500(22)00167-4

Cited by 12 publications

(4 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…are inherently uncertain, and effectively communicating forecast uncertainty to decision-makers and the public is critical for transparency, yet remains challenging 31,44 . In consideration of these needs, PandemicLLMs were finetuned to predict future pandemic trends by generating probabilities within defined categories, where the probability of the predicted category indicates the models' confidence in their predictions.…”

Section: Discussionmentioning

confidence: 99%

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

Yang,

Du,

Zhao

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Forecasting the short-term spread of an ongoing disease outbreak is a formidable challenge due to the complexity of contributing factors, some of which can be characterized through interlinked, multi-modality variables such as epidemiological time series data, viral biology, population demographics, and the intersection of public policy and human behavior. Existing forecasting model frameworks struggle with the multifaceted nature of relevant data and robust results translation, which hinders their performances and the provision of actionable insights for public health decision-makers. Our work introduces PandemicLLM, a novel framework with multi-modal Large Language Models (LLMs) that reformulates real-time forecasting of disease spread as a text reasoning problem, with the ability to incorporate real-time, complex, non-numerical information -- such as textual policies and genomic surveillance data -- previously unattainable in traditional forecasting models. This approach, through a unique AI-human cooperative prompt design and time series representation learning, encodes multi-modal data for LLMs. By redefining the forecasting process as an ordinal classification task, PandemicLLM yields more robust and trustworthy predictions, facilitating public health decision-making. The model is applied to the COVID-19 pandemic, and trained to utilize textual public health policies, genomic surveillance, spatial, and epidemiological time series data, and is subsequently tested across all 50 states of the U.S. for a duration of 16 weeks. Empirically, PandemicLLM is shown to be a high-performing pandemic forecasting framework that effectively captures the impact of emerging variants and can provide timely and accurate predictions. The proposed PandemicLLM opens avenues for incorporating various pandemic-related data in heterogeneous formats and exhibits performance benefits over existing models. This study illuminates the potential of adapting LLMs and representation learning to enhance pandemic forecasting, illustrating how AI innovations can strengthen pandemic responses and crisis management in the future.

show abstract

Section: Discussionmentioning

confidence: 99%

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

Yang,

Du,

Zhao

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Testing policies, which vary across countries over time, may also impact the assessment of

. Thus, relying solely on daily case numbers as a measure of the disease spread is problematic, especially when comparing disease spreads among different countries [26] . In contrast to

is an estimation of the rate of encounters between infected and non-infected individuals that would have resulted in an infection without vaccinations.…”

Section: Methodsmentioning

confidence: 99%

MAM: Flexible Monte-Carlo Agent based model for modelling COVID-19 spread

De-Leon

Aran

2023

Journal of Biomedical Informatics

View full text Add to dashboard Cite

“…Agent-based models were also widely used, employing computer simulations to generate a virtual environment where individuals follow defined rules 8 . However, not all modelling and evidence were likely adequate, effectively communicated, or effectively used by decision-makers [9][10][11] .…”

Section: Introductionmentioning

confidence: 99%

Lessons Learned From Covid-19 Modelling Efforts for Policy Decision-Making in Lower- And Middle-Income Countries

Owek,

Guleid,

Maluni

et al. 2024

Preprint

View full text Add to dashboard Cite

Introduction The COVID-19 pandemic had devastating health and socio-economic effects, partly due to mitigating policy choices. There is little evidence of approaches that guided policy decisions in settings that had limited modelling capacity pre-pandemic. We sought to identify knowledge translation mechanisms, enabling factors, and structures needed to translate modelled evidence to policy decisions effectively. Methods We utilised convergent mixed methods in a participatory action approach, with quantitative data from a survey and qualitative data from a scoping review, in-depth interviews, and workshop notes. Participants included researchers and policy actors involved in COVID-19 evidence generation and decision-making. They were mostly from lower- and middle-income countries (LMICs) in Africa, Southeast Asia, and Latin America. Quantitative and qualitative data integration occurred during data analysis through triangulation and during reporting in a narrative synthesis. Results We engaged 147 researchers and 57 policy actors from 28 countries. We found that the strategies required to use modelling evidence effectively include capacity building of modelling expertise and communication, improved data infrastructure, sustained funding, and dedicated knowledge translation platforms. The common knowledge translation mechanisms used during the pandemic included policy briefs, face-to-face debriefings, and dashboards. Some enabling factors for knowledge translation comprised solid relationships and open communication between researchers and policymakers, credibility of researchers, co-production of policy questions, and embedding researchers in policymaking spaces. Barriers included competition among modellers, negative attitude of policymakers towards research, political influences and demand for quick outputs. Conclusion Our findings led to the co-development of a knowledge translation framework useful in various settings to guide decision-making, especially for public health emergencies. Furthermore, we provide a contextualised understanding of knowledge translation for LMICs during the COVID-19 pandemic. Finally, we share key lessons on how knowledge translation from mathematical modelling complements the broader learning agenda related to pandemic preparedness and long-term investments in evidence-to-policy translation.

show abstract

Real-time COVID-19 forecasting: challenges and opportunities of model performance and translation

Cited by 12 publications

References 12 publications

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

Advancing Real-time Pandemic Forecasting Using Large Language Models: A COVID-19 Case Study

MAM: Flexible Monte-Carlo Agent based model for modelling COVID-19 spread

Lessons Learned From Covid-19 Modelling Efforts for Policy Decision-Making in Lower- And Middle-Income Countries

Contact Info

Product

Resources

About