Benchmarking Variational AutoEncoders on cancer transcriptomics data

Abstract

Deep generative models, such as variational autoencoders (VAE), have gained increasing attention in computational biology due to their ability to capture complex data manifolds which subsequently can be used to achieve better performance in downstream tasks, such as cancer type prediction or subtyping of cancer. However, these models are difficult to train due to the large number of hyperparameters that need to be tuned. To get a better understanding of the importance of the different hyperparameters, we examined six different VAE models when trained on TCGA transcriptomics data and evaluated on the downstream task of cluster agreement with cancer subtypes. We studied the effect of the latent space dimensionality, learning rate, optimizer and initialization on the quality of subsequent clustering of the TCGA samples. We found β-TCVAE and DIP-VAE to have a sensitive to hyperparameters selection. Based on these experiments, we derived recommendations for selecting the different hyperparameters settings. In addition, we examined whether the learned latent spaces capture biologically relevant information. Hereto, we correlated the different representations with various data characteristics such as age, days to metastasis, immune infiltration, and mutation signatures. We found that for all models the latent factors, in general, do not uniquely correlate with one of the data characteristics even for models specifically designed for disentanglement

An in-depth comparison of linear and non-linear joint embedding methods for bulk and single-cell multi-omics

“…The performances of the methods were estimated using another held-out test set comprising previously unseen data points. If, for some reason, the validation loss is not predictive of the performance at a specific downstream task -as we have previously shown can be the case for VAEs trained on RNA-Seq data [28] -then the chosen hyperparameters for a given model might not be the optimal ones for the downstream task. Our results here hint that this might be the case in our setting too.…”

Novel multi-omics deconfounding variational autoencoders can obtain meaningful disease subtyping

“…The identification of disease subtypes requires performing a clustering algorithm at some point. Even though iterative training of the clustering in a joint autoencoder loss function can overcome inconsistencies between training and downstream clustering performance [8, 19, 21], we chose for a decoupled strategy. This was to 1) avoid having too many terms in loss function to confuse training, and 2) reduce computation time and initialization settings with iteratively training clustering in a joint loss function.…”

“…In our study, clustering is done as an additional step on the latent features to guarantee the optimal performance of the deconfounding autoencoder. However, other researchers have pointed out the inconsistency between validation loss and downstream clustering performance [34] and proposed to iteratively train the clustering in a joint loss function with the autoencoder [35, 36]. We chose the standalone clustering strategy over the iterative one because 1) we want to avoid having too many terms in the loss function to confuse the training process and 2) the iterative clustering relies on a good initialization and is computationally expensive.…”

An in-depth comparison of linear and non-linear joint embedding methods for bulk and single-cell multi-omics

“…The performances of the methods were estimated using another held-out test set comprising previously unseen data points. If, for some reason, the validation loss is not predictive of the performance at a specific downstream task – as we have previously shown can be the case for VAEs trained on RNA-Seq data [ 30 ] – then the chosen hyperparameters for a given model might not be the optimal ones for the downstream task. Our results here hint that this might be the case in our setting too.…”