Genomic Surveillance of COVID-19 Variants with Language Models and Machine Learning

Abstract

The global efforts to control COVID-19 are threatened by the rapid emergence of novel variants that may display undesirable characteristics such as immune escape or increased pathogenicity. The current approaches to genomic surveillance do not allow early prediction of emerging variations. Here, we derive Dimensions of Concern (DoC) in the latent space of SARS-CoV-2 mutations and demonstrate their potential to provide a lead time for predicting the increase of new cases in 9 countries across the globe. We learned unsupervised word embeddings from 3,09,060 spike protein coding sequences deposited on GISAID database until April, 2021. We discovered that "blips" in the latent dimensions of embeddings are associated with mutations. We modeled the temporal occurrence of blips and their relationships with the number of new cases in the following months for these countries. Certain dimensions demonstrated a consistent leading relationship between the occurrence of blips and the number of new cases in the following months, thus labeled as potential Dimensions of Concern, DoCs. We validated the predictive importance of DoCs by performing Random Forest-based feature selection and modeling in a temporally split training, validation, testing regime. Twelve dimensions achieved statistical significance and achieved an R-squared of 37% for prediction of number of new cases in the following month. Biological exploration of DoCs revealed that dimensions 3 and 12 captures 3-mers CGG, ACG and CAC that are associated with known variants L452R, K417T and Q677H respectively. Learning and tracking DoCs is extensible to related challenges such as pandemic preparedness, immune escape, pathogenicity modeling and antimicrobial resistance.