Biclustering by sparse canonical correlation analysis

Pimentel, Harold; Hu, Zhiyue Tom; Huang, Haiyan

doi:10.1007/s40484-017-0127-0

“…That is, one should not interpret parts of a model which are not stable to appropriate perturbations to the model and data. This is demonstrated through examples in the text (21,24,25).…”

Section: Introductionmentioning

confidence: 99%

Definitions, methods, and applications in interpretable machine learning

Murdoch

¹

,

Singh

²

,

Kumbier

³

et al. 2019

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has led to considerable confusion about the notion of interpretability. In particular, it is unclear how the wide array of proposed interpretation methods are related and what common concepts can be used to evaluate them. We aim to address these concerns by defining interpretability in the context of machine learning and introducing the predictive, descriptive, relevant (PDR) framework for discussing interpretations. The PDR framework provides 3 overarching desiderata for evaluation: predictive accuracy, descriptive accuracy, and relevancy, with relevancy judged relative to a human audience. Moreover, to help manage the deluge of interpretation methods, we introduce a categorization of existing techniques into model-based and post hoc categories, with subgroups including sparsity, modularity, and simulatability. To demonstrate how practitioners can use the PDR framework to evaluate and understand interpretations, we provide numerous real-world examples. These examples highlight the often underappreciated role played by human audiences in discussions of interpretability. Finally, based on our framework, we discuss limitations of existing methods and directions for future work. We hope that this work will provide a common vocabulary that will make it easier for both practitioners and researchers to discuss and choose from the full range of interpretation methods.interpretability | machine learning | explainability | relevancy

show abstract

“…That is, one should not interpret parts of a model which are not stable to appropriate perturbations to the model and data. This is demonstrated through examples in the text (20,23,24).…”

Section: Stabilitymentioning

confidence: 99%

“…For instance, one recent study (23) uses a biclustering approach based on sparse canonical correlation analysis (SCCA) to identify interactions among genomic expression features in Drosophila melanogaster (fruit flies) and Caenorhabditis elegans (roundworms). Sparsity penalties enable key interactions among features to be summarized in heatmaps which contain few enough variables for a human to analyze.…”

Section: Model-based Interpretabilitymentioning

confidence: 99%

Interpretable machine learning: definitions, methods, and applications

Murdoch,

Singh,

Kumbier

et al. 2019

Preprint

View full text Add to dashboard Cite

Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has led to considerable confusion about the notion of interpretability. In particular, it is unclear how the wide array of proposed interpretation methods are related, and what common concepts can be used to evaluate them.We aim to address these concerns by defining interpretability in the context of machine learning and introducing the Predictive, Descriptive, Relevant (PDR) framework for discussing interpretations. The PDR framework provides three overarching desiderata for evaluation: predictive accuracy, descriptive accuracy and relevancy, with relevancy judged relative to a human audience. Moreover, to help manage the deluge of interpretation methods, we introduce a categorization of existing techniques into model-based and post-hoc categories, with sub-groups including sparsity, modularity and simulatability. To demonstrate how practitioners can use the PDR framework to evaluate and understand interpretations, we provide numerous real-world examples. These examples highlight the often under-appreciated role played by human audiences in discussions of interpretability. Finally, based on our framework, we discuss limitations of existing methods and directions for future work. We hope that this work will provide a common vocabulary that will make it easier for both practitioners and researchers to discuss and choose from the full range of interpretation methods. * For clarity, throughout the paper we use the term to refer to both machine-learning models and algorithms.accuracy, and relevancy, where relevancy is judged by a human audience. Using these terms, we categorize a broad range of existing methods, all grounded in real-world examples † . In doing so, we provide a common vocabulary for researchers and practitioners to use in evaluating and selecting interpretation methods. We then show how our work enables a clearer discussion of open problems for future research.A. Defining interpretable machine learning. On its own, interpretability is a broad, poorly defined concept. Taken to its full generality, to interpret data means to extract information (of some form) from it. The set of methods falling under this umbrella spans everything from designing an initial experiment to visualizing final results. In this overly general form, interpretability is not substantially different from the established concepts of data science and applied statistics.Instead of general interpretability, we focus on the use of interpretations in the context of ML as part of the larger datascience life cycle. We define interpretable machine learning as the use of machine-learning models for the extraction of relevant knowledge about domain relationships contained in data. Here, we view knowledge as being relevant if it pr...

show abstract

“…When d is set to a dimension satisfying d < D x , D y , these models find dimensional reduced representations that may be useful for clustering, classification or manifold learning in many applications. For example, in biology [8], neuroscience [9], medicine [10], and engineering [11]. One key limitations of these models is that they typically require more samples than features, i.e.…”

Section: Introductionmentioning

confidence: 99%

$\ell_0$-based Sparse Canonical Correlation Analysis

Lindenbaum¹,

Salhov²,

Averbuch³

et al. 2020

Preprint

View full text Add to dashboard Cite

Canonical Correlation Analysis (CCA) models can extract informative correlated representations from multimodal unlabelled data. Despite their success, CCA models may break if the number of variables exceeds the number of samples. We propose Deep Gated-CCA, a method for learning correlated representations based on a sparse subset of variables from two observed modalities. The proposed procedure learns two non-linear transformations and simultaneously gates the input variables to identify a subset of most correlated variables. The non-linear transformations are learned by training two neural networks to maximize a shared correlation loss defined based on their outputs. Gating is obtained by adding an approximate 0 regularization term applied to the input variables. This approximation relies on a recently proposed continuous Gaussian based relaxation for Bernoulli variables which act as gates. We demonstrate the efficacy of the method using several synthetic and real examples. Most notably, the method outperforms other linear and non-linear CCA models.

show abstract

Biclustering by sparse canonical correlation analysis

Cited by 5 publications

References 29 publications

Definitions, methods, and applications in interpretable machine learning

Definitions, methods, and applications in interpretable machine learning

Interpretable machine learning: definitions, methods, and applications

$\ell_0$-based Sparse Canonical Correlation Analysis

Contact Info

Product

Resources

About