Evaluating performance of existing computational models in predicting CD8+ T cell pathogenic epitopes and cancer neoantigens

Abstract

T cell recognition of a cognate peptide–major histocompatibility complex (pMHC) presented on the surface of infected or malignant cells is of the utmost importance for mediating robust and long-term immune responses. Accurate predictions of cognate pMHC targets for T cell receptors would greatly facilitate identification of vaccine targets for both pathogenic diseases and personalized cancer immunotherapies. Predicting immunogenic peptides therefore has been at the center of intensive research for the past decades but has proven challenging. Although numerous models have been proposed, performance of these models has not been systematically evaluated and their success rate in predicting epitopes in the context of human pathology has not been measured and compared. In this study, we evaluated the performance of several publicly available models, in identifying immunogenic CD8+ T cell targets in the context of pathogens and cancers. We found that for predicting immunogenic peptides from an emerging virus such as severe acute respiratory syndrome coronavirus 2, none of the models perform substantially better than random or offer considerable improvement beyond HLA ligand prediction. We also observed suboptimal performance for predicting cancer neoantigens. Through investigation of potential factors associated with ill performance of models, we highlight several data- and model-associated issues. In particular, we observed that cross-HLA variation in the distribution of immunogenic and non-immunogenic peptides in the training data of the models seems to substantially confound the predictions. We additionally compared key parameters associated with immunogenicity between pathogenic peptides and cancer neoantigens and observed evidence for differences in the thresholds of binding affinity and stability, which suggested the need to modulate different features in identifying immunogenic pathogen versus cancer peptides. Overall, we demonstrate that accurate and reliable predictions of immunogenic CD8+ T cell targets remain unsolved; thus, we hope our work will guide users and model developers regarding potential pitfalls and unsettled questions in existing immunogenicity predictors.

Can we predict T cell specificity with digital biology and machine learning?

“…Many groups have attempted to bypass this complexity by predicting antigen immunogenicity independent of the TCR 14 , as a direct mapping from peptide sequence to T cell activation. However, similar limitations have been encountered for those models as we have described for specificity inference.…”

“…Although great strides have been made in improving prediction of antigen processing and presentation for common HLA alleles, the nature and extent to which presented peptides trigger a T cell response are yet to be elucidated 13 . A significant gap also remains for the prediction of T cell activation for a given peptide 14 , 15 , and the parameters that influence pathological peptide or neoantigen immunogenicity remain under intense investigation 16 . We believe that only by integrating knowledge of antigen presentation, TCR recognition, context-dependent activation and effector function at the cell and tissue level will we fully realize the benefits to fundamental and translational science (Box 2 ).…”

A systems approach evaluating the impact of SARS-CoV-2 variant of concern mutations on CD8+ T cell responses

“…While in silico models that predict antigen presentation can be highly accurate (such as netMHCpan), state-of-the-art models predicting the subset of HLA ligands that then invoke T cell responses possess limited accuracy [ 39 ]. We recently developed TRAP (T cell recognition potential of HLA-I presented peptides) [ 33 ], a convolutional neural network (CNN) model that offers improved predictions of T cell recognition potential of HLA-I presented 9- and 10-mer peptides.…”

“…In that light, in silico studies e.g., the work by Nersisyan et al [ 29 ] compared all theoretical HLA ligands across specific VOCs and concluded that T cell responses to Omicron were likely to be maintained effectively. However, not all HLA ligands can invoke T cell responses [ 38 , 39 ]. Furthermore, studies such as those of Naranbhai et al [ 17 ] and Reynolds et al [ 18 ] observed considerable numbers of patients with impaired T cell responses to Omicron infection [ 57 ].…”

Discovering and overcoming the bias in neoantigen identification by unified machine learning models

“…In recent years, a plethora of machine learning methods have been developed to predict each of the above three processes related to neoantigens, i.e., binding [7, 8, 9, 10, 11, 12, 13, 14], presentation [15, 16, 17, 18, 19, 20] and immunogenicity [21, 22, 23, 24, 25, 26], each of which contributes to neoantigen identification from different perspectives. However, despite the progress made in both scientific research and clinical therapeutics [27, 28, 29, 30, 31], the overall accuracy of neoantigen identification remains far from satisfactory [32, 33, 34]. Only 6% of the predicted epitopes from the global consortium TESLA were positive when tested with immunogenicity [33].…”