Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model

Bai, Junwen; Kong, Shufeng; Gomes, Carla P.

doi:10.24963/ijcai.2020/595

“…The integration of AI techniques in the present work builds upon the foundation of our prior work developing general multi-label classification and multi-target regression approaches that were initially motivated by ecology applications. These prior works demonstrated the utility of multivariate Gaussian used for pairwise correlation learning, 36 model alignment with a VAE during training, 37,39 and high-order correlation learning via an attention graph neural network. 38 Our careful crafting of model architecture is particularly motivated by the need to make predictions in new composition spaces.…”

Section: Discussionmentioning

confidence: 99%

“…We tackle this challenge with correlation learning, which has been demonstrated to enhance multi-label classification and multi-target regression. [36][37][38][39] Since the multiple properties being predicted may not be explicitly correlated, we developed a framework to learn correlations in latent embeddings of the multiple properties.…”

Section: Introductionmentioning

confidence: 99%

Materials Representation and Transfer Learning for Multi-Property Prediction

Kong¹,

Guevarra²,

Gomes³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements, as well as the relationships among multiple properties, to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates (i) prediction using only a material’s composition, (ii) learning and exploitation of correlations among target properties in multitarget regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 3-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data is available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with Transfer learning (H-CLMP(T)) wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well-suited for multi-target regression across the physical sciences.

show abstract

“…The integration of AI techniques in the present work build upon the foundation of our prior work developing general multi-label classification and and multi-target regression approaches that were initially motivated by ecology applications. These prior works demonstrated the utility of multivariate Gaussian used for pairwise correlation learning, 35 model alignment with a VAE during training, 36,38 and high-order correlation learning via an attention graph neural network. 37 Our careful crafting of model architecture is particularly motivated by the need to make predictions in new composition spaces.…”

Section: Discussionmentioning

confidence: 99%

“…We tackle this challenge with correlation learning, which has been demonstrated to enhance multilabel classification and multi-target regression. [35][36][37][38] Since the multiple properties being predicted may not be explicitly correlated, we developed a framework to learn correlations in latent embeddings of the multiple properties.…”

Section: Introductionmentioning

confidence: 99%

Materials Representation and Transfer Learning for Multi-Property Prediction

Kong¹,

Guevarra²,

Gomes³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements, as well as the relationships among multiple properties, to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates (i) prediction using only a material’s composition, (ii) learning and exploitation of correlations among target properties in multitarget regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 3-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data is available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with Transfer learning (H-CLMP(T)) wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well-suited for multi-target regression across the physical sciences.

show abstract

“…Even if the corresponding feature and label codes are close, we cannot guarantee the decoded targets are similar. To address this concern, MPVAE (Bai, Kong, and Gomes 2020) proposes to replace the deterministic latent space with a probabilistic subspace under a variational autoencoder (VAE) framework. The Gaussian latent spaces are aligned with KL-divergence, and the sampling process enforces smoothness.…”

Section: Introductionmentioning

confidence: 99%

Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification

Bai¹,

Kong

²

,

Gomes³

2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

Multi-label classification (MLC) is a prediction task where each sample can have more than one label. We propose a novel contrastive learning boosted multi-label prediction model based on a Gaussian mixture variational autoencoder (C-GMVAE), which learns a multimodal prior space and employs a contrastive loss. Many existing methods introduce extra complex neural modules to capture the label correlations, in addition to the prediction modules. We found that by using contrastive learning in the supervised setting, we can exploit label information effectively, and learn meaningful feature and label embeddings capturing both the label correlations and predictive power, without extra neural modules. Our method also adopts the idea of learning and aligning latent spaces for both features and labels. C-GMVAE imposes a Gaussian mixture structure on the latent space, to alleviate posterior collapse and over-regularization issues, in contrast to previous works based on a unimodal prior. C-GMVAE outperforms existing methods on multiple public datasets and can often match other models' full performance with only 50% of the training data. Furthermore, we show that the learnt embeddings provide insights into the interpretation of labellabel interactions.

show abstract

Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model

Cited by 29 publications

References 18 publications

Materials Representation and Transfer Learning for Multi-Property Prediction

Materials Representation and Transfer Learning for Multi-Property Prediction

Materials Representation and Transfer Learning for Multi-Property Prediction

Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification

Contact Info

Product

Resources

About