SNVformer: An Attention-based Deep Neural Network for GWAS Data

Abstract

Despite being the widely-used gold standard for linking common genetic variations to phenotypes and disease, genome-wide association studies (GWAS) suffer major limitations, partially attributable to the reliance on simple, typically linear, models of genetic effects. More elaborate methods, such as epistasis-aware models, typically struggle with the scale of GWAS data. In this paper, we build on recent advances in neural networks employing Transformer-based architectures to enable such models at a large scale. As a first step towards replacing linear GWAS with a more expressive approximation, we demonstrate prediction of gout, a painful form of inflammatory arthritis arising when monosodium urate crystals form in the joints under high serum urate conditions, from Single Nucleotide Variants (SNVs) using a scalable (long input) variant of the Transformer architecture. Furthermore, we show that sparse SNVs can be efficiently used by these Transformer-based networks without expanding them to a full genome. By appropriately encoding SNVs, we are able to achieve competitive initial performance, with an AUROC of 83% when classifying a balanced test set using genotype and demographic information. Moreover, the confidence with which the network makes its prediction is a good indication of the prediction accuracy. Our results indicate a number of opportunities for extension, enabling full genome-scale data analysis using more complex and accurate genotype-phenotype association models.

Accuracy and Scalability of Machine Learning Methods for Genotype-Phenotype Association Data

“…We use a variation of the SNVformer architecture [11], with additions to the SNV encoding. We tested several variations of the original SNVformer encoding (see Appendix F.2) the best-performing variation is described here.…”

Machine Learning to Advance Human Genome-Wide Association Studies

“…The complexities of genetic information pose unique challenges, such as high dimensionality and the need for significant computational power, which have so far hindered the widespread adoption of foundation models in this area with relatively few publications applying basic concepts of foundation models to genomic data [104,[106][107][108]. For example, Santiesteban et al [109] showed that foundation models combining transcriptomics and histopathology data through self-supervised learning significantly improve survival prediction.…”

A genotype-phenotype transformer to assess and explain polygenic risk

“…Recent developments in deep learning [13][14][15] offer significant opportunities to advance the PRS framework [16][17][18][19] , as they have the capacity to model both complex epistatic interactions and knowledge of molecular mechanisms. Among these, the Transformer has shown strong potential to address longstanding problems in the biomedical sciences, including prediction of 3D protein structures [20][21][22] , biomedical image analysis 23 , inference of gene expression from genome sequence [24][25][26] , and mapping a sequence of SNPs to a predicted phenotype 27,28 . The Transformer architecture is known for its central use of an "attention" mechanism 29 , an operation that dynamically computes the importance of each input element relative to others, enabling the model to focus on the most relevant features [29][30][31] .…”