Estimating a Brain Network Predictive of Stress and Genotype with Supervised Autoencoders

Talbot, Austin; Dunson, David B.; Dzirasa, Kafui; Carlson, David

doi:10.48550/arxiv.2004.05209

Cited by 3 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Derivations of analytic solutions can be referenced in Appendix A. Further discussion on the properties of these factor models can be found in [23] and [24].…”

Section: Augmentedpca Factor Modelsmentioning

confidence: 99%

AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models

Carson¹,

Talbot²,

Carlson³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep autoencoders are often extended with a supervised or adversarial loss to learn latent representations with desirable properties, such as greater predictivity of labels and outcomes or fairness with respects to a sensitive variable. Despite the ubiquity of supervised and adversarial deep latent factor models, these methods should demonstrate improvement over simpler linear approaches to be preferred in practice. This necessitates a reproducible linear analog that still adheres to an augmenting supervised or adversarial objective. We address this methodological gap by presenting methods that augment the principal component analysis (PCA) objective with either a supervised or an adversarial objective and provide analytic and reproducible solutions. We implement these methods in an open-source Python package, AugmentedPCA, that can produce excellent real-world baselines. We demonstrate the utility of these factor models on an open-source, RNA-seq cancer gene expression dataset, showing that augmenting with a supervised objective results in improved downstream classification performance, produces principal components with greater class fidelity, and facilitates identification of genes aligned with the principal axes of data variance with implications to development of specific types of cancer.

show abstract

“…Derivations of analytic solutions can be referenced in Appendix A. Further discussion on the properties of these factor models can be found in [23] and [24].…”

Section: Augmentedpca Factor Modelsmentioning

confidence: 99%

AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models

Carson¹,

Talbot²,

Carlson³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Once the prior distribution of the latent factors and the conditional distributions have been defined, statistical estimation is straightforward by maximum likelihood or Bayesian methods [27]. However, it has been demonstrated that joint models suffer under model misspecification, particularly when the number of learned factors is less than the true latent dimensionality [23,28]. Furthermore, the variance of the outcomes is often small relative to the variance of the observations, which leads to the outcome being poorly characterized [13].…”

Section: Related Workmentioning

confidence: 99%

“…To address some of these limitations, SAEs can be used to effectively approximate traditional factor models while including a predictive term. SAEs have been shown to ameliorate some concerns about model misspecficiation [28]. SAEs have been shown to give gains in many predictive applications [18,16].…”

Section: Related Workmentioning

confidence: 99%

“…If our only objective were to obtain a generative model of x, we could simply parameterize q φ with a flexible neural network and maximize the evidence lower bound (ELBO) used in VAEs. However, focusing exclusively on the generative model often yields poor predictions of y [28]. This motivates the inclusion of a supervised loss, forming the SVAE objective…”

Section: A Supervised Variational Autoencodersmentioning

confidence: 99%

“…If λ = 0, then (1) reduces to the standard VAE objective. In practice, λ is usually set to a fairly high value to emphasize prediction, as the variance of y is often substantially outweighed by the variance of x [28].…”

Section: A Supervised Variational Autoencodersmentioning

confidence: 99%

See 2 more Smart Citations

Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility

Tu¹,

Talbot

Gallagher³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Probabilistic generative models are attractive for scientific modeling because their inferred parameters can be used to generate hypotheses and design experiments. This requires that the learned model provide an accurate representation of the input data and yield a latent space that effectively predicts outcomes relevant to the scientific question. Supervised Variational Autoencoders (SVAEs) have previously been used for this purpose, where a carefully designed decoder can be used as an interpretable generative model while the supervised objective ensures a predictive latent representation. Unfortunately, the supervised objective forces the encoder to learn a biased approximation to the generative posterior distribution, which renders the generative parameters unreliable when used in scientific models. This issue has remained undetected as reconstruction losses commonly used to evaluate model performance do not detect bias in the encoder. We address this previously-unreported issue by developing a second order supervision framework (SOS-VAE) that influences the decoder to induce a predictive latent representation. This ensures that the associated encoder maintains a reliable generative interpretation. We extend this technique to allow the user to trade-off some bias in the generative parameters for improved predictive performance, acting as an intermediate option between SVAEs and our new SOS-VAE. We also use this methodology to address missing data issues that often arise when combining recordings from multiple scientific experiments. We demonstrate the effectiveness of these developments using synthetic data and electrophysiological recordings with an emphasis on how our learned representations can be used to design scientific experiments.

show abstract

Estimating a Brain Network Predictive of Stress and Genotype with Supervised Autoencoders

Cited by 3 publications

References 0 publications

AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models

AugmentedPCA: A Python Package of Supervised and Adversarial Linear Factor Models

Supervising the Decoder of Variational Autoencoders to Improve Scientific Utility

Prenatal environmental stressors impair postnatal microglia function and adult behavior in males

Contact Info

Product

Resources

About