MOT: a Multi-Omics Transformer for multiclass classification tumour types predictions

Abstract

Motivation Breakthroughs in high-throughput technologies and machine learning methods have enabled the shift towards multi-omics modelling as the preferred means to understand the mechanisms underlying biological processes. Machine learning enables and improves complex disease prognosis in clinical settings. However, most multi-omic studies primarily use transcriptomics and epigenomics due to their over-representation in databases and their early technical maturity compared to others omics. For complex phenotypes and mechanisms, not leveraging all the omics despite their varying degree of availability can lead to a failure to understand the underlying biological mechanisms and leads to less robust classifications and predictions. Results We proposed MOT (Multi-Omic Transformer), a deep learning based model using the transformer architecture, that discriminates complex phenotypes (herein cancer types) based on five omics data types: transcriptomics (mRNA and miRNA), epigenomics (DNA methylation), copy number variations (CNVs), and proteomics. This model achieves an F1-score of $98.37\%$ among 33 tumour types on a test set without missing omics views and an F1-score of $96.74\%$ on a test set with missing omics views. It also identifies the required omic type for the best prediction for each phenotype and therefore could guide clinical decision-making when acquiring data to confirm a diagnostic. The newly introduced model can integrate and analyze five or more omics data types even with missing omics views and can also identify the essential omics data for the tumour multiclass classification tasks. It confirms the importance of each omic view. Combined, omics views allow a better differentiation rate between most cancer diseases. Our study emphasized the importance of multi-omic data to obtain a better multiclass cancer classification. Availability and implementation: MOT source code is available at \url{https://github.com/dizam92/multiomic_predictions}.

Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data

“…However, incorporating multi-omics data and their crosstalk information in a Transformer is very challenging: when processing multi-omics data, the multi-modal features are usually multiplied by tens of thousands of genes, producing an extremely long input that is not acceptable by a common Transformer model (usually <512 words). Meanwhile, certain embedding methods for biological data, such as discretization and linear transformation, were introduced in the previous Transformer models ( Osseni et al 2022 , Cui et al 2023 , Theodoris et al 2023 ), while biological information was largely lost during these kinds of embedding.…”

Pathformer: a biological pathway informed Transformer integrating multi-omics data for disease diagnosis and prognosis

“…However, incorporating multi-omics data and their crosstalk information in a Transformer is very challenging: when processing multi-omics data, the multi-modal features are usually multiplied by the number of genes (tens of thousands), producing an extremely long input that is not acceptable by a common Transformer model (usually less than 512). Meanwhile, certain embedding methods for biological data, such as discretization and linear transformation, were introduced in the previous Transformer models [17][18][19] , while biological information was largely lost during these kinds of embedding.…”

“…The crisscross attention mechanism of Transformer would be very useful to capture the crosstalk information 15 . However, the current embedding methods for biological data in Transformer models [16][17][18] , such as discretization and linear transformation, usually missed the biological information. Meanwhile, multi-modal features were typically multiplied by the number of genes, resulting in long input sequences, which could strain available memory resources or necessitate a feature selection step 18,19 .…”