Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt

Mindermann, Sören; Brauner, Jan Markus; Razzak, Muhammed; Sharma, Mrinank; Kirsch, Andreas; Xu, Winnie; Höltgen, Benedikt; Gomez, Aidan N.; Morisot, Adrien; Farquhar, Sebastian; Gal, Yarin

doi:10.48550/arxiv.2206.07137

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the training procedure takes into account variation from 70 individuals, WGS-paired RNA-seq data is available for many more GTEx samples, and can be supplemented with additional datasets 48 . To take into account such a large amount of data, methods have been developed to prioritize the most informative training points 49 , allowing the training procedure to scale and effectively learn from extremely large datasets. To explore improved prediction of differences between individuals, a contrastive training objective can be used 50,51,52 and predictions can be made for the difference in expression between two haplotypes 53 .…”

Section: Discussionmentioning

confidence: 99%

An RNA foundation model enables discovery of disease mechanisms and candidate therapeutics

Celaj,

Gao,

Lau

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurately modeling and predicting RNA biology has been a long-standing challenge, bearing significant clinical ramifications for variant interpretation and the formulation of tailored therapeutics. We describe a foundation model for RNA biology, “BigRNA”, which was trained on thousands of genome-matched datasets to predict tissue-specific RNA expression, splicing, microRNA sites, and RNA binding protein specificity from DNA sequence. Unlike approaches that are restricted to missense variants, BigRNA can identify pathogenic non-coding variant effects across diverse mechanisms, including polyadenylation, exon skipping and intron retention. BigRNA accurately predicted the effects of steric blocking oligonucleotides (SBOs) on increasing the expression of 4 out of 4 genes, and on splicing for 18 out of 18 exons across 14 genes, including those involved in Wilson disease and spinal muscular atrophy. We anticipate that BigRNA and foundation models like it will have widespread applications in the field of personalized RNA therapeutics.

show abstract