Factorized embeddings learns rich and biologically meaningful embedding spaces using factorized tensor decomposition

Abstract

Motivation The recent development of sequencing technologies revolutionized our understanding of the inner workings of the cell as well as the way disease is treated. A single RNA sequencing (RNA-Seq) experiment, however, measures tens of thousands of parameters simultaneously. While the results are information rich, data analysis provides a challenge. Dimensionality reduction methods help with this task by extracting patterns from the data by compressing it into compact vector representations. Results We present the factorized embeddings (FE) model, a self-supervised deep learning algorithm that learns simultaneously, by tensor factorization, gene and sample representation spaces. We ran the model on RNA-Seq data from two large-scale cohorts and observed that the sample representation captures information on single gene and global gene expression patterns. Moreover, we found that the gene representation space was organized such that tissue-specific genes, highly correlated genes as well as genes participating in the same GO terms were grouped. Finally, we compared the vector representation of samples learned by the FE model to other similar models on 49 regression tasks. We report that the representations trained with FE rank first or second in all of the tasks, surpassing, sometimes by a considerable margin, other representations. Availability and implementation A toy example in the form of a Jupyter Notebook as well as the code and trained embeddings for this project can be found at: https://github.com/TrofimovAssya/FactorizedEmbeddings. Supplementary information Supplementary data are available at Bioinformatics online.

Evaluation of input data modality choices on functional gene embeddings

“…To cope with dataset-specific missing values, we modified the deep tensor factorisation model by Trofimov et al. ( 6 ). We further generated two embeddings from the functional PPI network STRING ( 23 ) using graph embedding algorithms (Materials and Methods).…”

“…Studies have demonstrated that functional gene embeddings can be derived from a variety of data types reflecting gene function, including gene expression ( 4–6 ), CRISPR screens ( 7 ), protein sequence ( 8 ), protein–protein interaction networks ( 9–11 ), but also occurrences in scientific literature ( 12 ) and Gene Ontology ( 13 ) annotations ( 14 ). These studies have shown that the obtained embeddings have predictive power for a wide range of downstream prediction tasks including the prediction of disease-gene lists ( 9 , 12 ), Gene Ontology annotations ( 6 , 7 ), gene-phenotype associations ( 15 ), but also gene-gene interactions ( 5 , 11 , 14 ) and subcellular localizations of gene products ( 10 ). These results are appealing because they suggest that precomputed functional gene embeddings could become effective resources for integrating gene function information into machine learning models.…”

Baseline Acute Myeloid Leukemia Prognosis Models using Transcriptomic and Clinical Profiles by Studying the Impacts of Dimensionality Reductions and Gene Signatures on Cox-Proportional Hazard

“…Plus, PCA performs linear transformations, which can be possibly insufficient to model data. Neural networks such as t-SNE (Hinton & Van Der Maaten, 2008) and Factorized Embedding (Trofimov et al, 2020) can learn useful non-linear data-driven representations. Adaptation of these DNNs in practice to learn survival-directed representations with a CPH could theoretically improve performance.…”

Embeddings of genomic region sets capture rich biological associations in lower dimensions

“…In bioinformatics, embeddings have been used to represent DNA nucleotide sequences [15][16][17][18] , protein sequences 19 , genes 20 , and single-cell Hi-C data 21 , among others. Factorized tensor decomposition has also been used to achieve biologically meaningful representation for RNA-Seq data 22 . Most similar to our use case is the recently published Avocado method, which learns a dense representation for a region using a deep neural network trained on epigenome signal information 23 .…”