Closed-form supervised dimensionality reduction with generalized linear models

Rish, Irina; Grabarnik, Genady Ya.; Cecchi, Guillermo A.; Pereira, Francisco; Gordon, Geoffrey J.

doi:10.1145/1390156.1390261

Cited by 35 publications

(31 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiview learning methods are often used to learn a better classifier for multi-label classification -usually in text mining and image classification domains -based on correlation structures among the training data and the labels (Yu et al, 2006;Virtanen et al, 2011;Rish, Grabarnik, Cecchi, Pereira, & Gordon, 2008). However, in medical analysis and diagnosis, we meet two separate tasksthe association discovery between genetic variations and clinical traits, and the diagnosis on patients.…”

Section: Related Workmentioning

confidence: 99%

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Zhe

et al. 2016

jair

View full text Add to dashboard Cite

The analysis and diagnosis of Alzheimer's disease (AD) can be based on genetic variations, e.g., single nucleotide polymorphisms (SNPs) and phenotypic traits, e.g., Magnetic Resonance Imaging (MRI) features. We consider two important and related tasks: i) to select genetic and phenotypical markers for AD diagnosis and ii) to identify associations between genetic and phenotypical data. While previous studies treat these two tasks separately, they are tightly coupled because underlying associations between genetic variations and phenotypical features contain the biological basis for a disease. Here we present a new sparse Bayesian approach for joint association study and disease diagnosis. In this approach, common latent features are extracted from different data sources based on sparse projection matrices and used to predict multiple disease severity levels; in return, the disease status can guide the discovery of relationships between data sources. The sparse projection matrices not only reveal interactions between data sources but also select groups of biomarkers related to the disease. Moreover, to take advantage of the linkage disequilibrium (LD) measuring the non-random association of alleles, we incorporate a graph Laplacian type of prior in the model. To learn the model from data, we develop an efficient variational inference algorithm. Analysis on an imaging genetics dataset for the study of Alzheimer's Disease (AD) indicates that our model identifies biologically meaningful associations between genetic variations and MRI features, and achieves significantly higher accuracy for predicting ordinal AD stages than the competing methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Zhe

et al. 2016

jair

View full text Add to dashboard Cite

show abstract

“…[29] proposes a supervised probabilistic PCA and an efficient solution method, but the algorithm is developed only for real outputs. [21] formulates a supervised dimensionality reduction algorithm coupled with generalized linear models for binary classification and regression, and maximize a target function composed of input and output likelihood terms using an iterative algorithm.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Supervised Multilabel Learning with Coupled Embedding and Classification

Gönen

2012

Proceedings of the 2012 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Coupled training of dimensionality reduction and classification is proposed previously to improve the prediction performance for single-label problems. Following this line of research, in this paper, we introduce a novel Bayesian supervised multilabel learning method that combines linear dimensionality reduction with linear binary classification. We present a deterministic variational approximation approach to learn the proposed probabilistic model for multilabel classification. We perform experiments on four benchmark multilabel learning data sets by comparing our method with four baseline linear dimensionality reduction algorithms. Experiments show that the proposed approach achieves good performance values in terms of hamming loss, macro F1, and micro F1 on held-out test data. The low-dimensional embeddings obtained by our method are also very useful for exploratory data analysis.

show abstract

“…While unsupervised latent semantic models are popular for dimensionality reduction, their supervised counterparts are similar in spirit to supervised dimensionality reduction [19], which aims at finding a low-dimensional representation of data such that the supervision information can be well fitted by some predictive model that will be used to predict unlabeled data. However, single task supervised feature learning may be not reliable when supervision information is not enough.…”

Section: Introductionmentioning

confidence: 99%

Multi-task Semi-supervised Semantic Feature Learning for Classification

Zhuang

et al. 2012

2012 IEEE 12th International Conference on Data Mining

View full text Add to dashboard Cite

Multi-task learning has proven to be useful to boost the learning of multiple related but different tasks. Meanwhile, latent semantic models such as LSA and NMF are popular and effective methods to extract discriminative semantic features of high dimensional dyadic data. In this paper, we present a method to combine these two techniques together by introducing a new matrix tri-factorization based formulation for semi-supervised latent semantic learning, which can incorporate labeled information into traditional unsupervised learning of latent semantics. Our inspiration for multi-task semantic feature learning comes from two facts, i.e., 1) multiple tasks generally share a set of common latent semantics; and 2) a semantic usually has a stable indication of categories no matter which task it is from. Thus to make multiple tasks learn from each other we wish to share the associations between categories and those common semantics among tasks. Along this line, we propose a novel joint Nonnegative matrix tri-factorization framework with the aforesaid associations shared among tasks in the form of a semantic-category relation matrix. Our new formulation for multi-task learning can simultaneously learn (1) discriminative semantic features of each task; (2) predictive structure and categories of unlabeled data in each task; (3) common semantics shared among tasks and specific semantics exclusive to each task. We give alternating iterative algorithm to optimize our objective and theoretically show its convergence. Finally extensive experiments on text data along with the comparison with various baselines and three state-of-the-art multi-task learning algorithms demonstrate the effectiveness of our method.

show abstract

Closed-form supervised dimensionality reduction with generalized linear models

Cited by 35 publications

References 7 publications

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Bayesian Supervised Multilabel Learning with Coupled Embedding and Classification

Multi-task Semi-supervised Semantic Feature Learning for Classification

Contact Info

Product

Resources

About

Closed-form supervised dimensionality reduction with generalized linear models

Cited by 35 publications

References 7 publications

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Bayesian Supervised Multilabel Learning with Coupled Embedding and Classification

Multi-task Semi-supervised Semantic Feature Learning for Classification

Contact Info

Product

Resources

About

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning

Association Discovery and Diagnosis of Alzheimers Disease with Bayesian Multiview Learning