Relation-Aware Shared Representation Learning for Cancer Prognosis Analysis With Auxiliary Clinical Variables and Incomplete Multi-Modality Data

Ning, Zhenyuan; Du, Denghui; Tu, Chao; Feng, Qianjin

doi:10.1109/tmi.2021.3108802

Cited by 13 publications

(3 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Ning et al (2021a ) present a multi-constraint latent representation learning method called McLR to achieve promising performance of cancer prognosis by learning a common subspace. Besides, Ning et al (2021b ) introduce an impressive method called Relation-aware Shared Representation learning (RaSR) to unify both representation learning and prognosis modeling into a joint framework and improve the performance of cancer prognosis. The above studies confirm that the combination of multimodal data is very helpful to enhance the performance of survival prediction thus providing a solid foundation for further research.…”

Section: Introductionmentioning

confidence: 99%

CAMR: cross-aligned multimodal representation learning for cancer survival prediction

Shi

Wang

et al. 2023

Bioinformatics

View full text Add to dashboard Cite

Motivation Accurately predicting cancer survival is crucial for helping clinicians to plan appropriate treatments, which largely improves the life quality of cancer patients and spares the related medical costs. Recent advances of survival prediction methods suggest that integrating complementary information from different modalities, e.g., histopathological images and genomic data, plays a key role in enhancing predictive performance. Despite promising results obtained by existing multimodal methods, the disparate and heterogeneous characteristics of multimodal data cause so-called modality gap problem, which brings in dramatically diverse modality representations in feature space. Consequently, detrimental modality gaps make it difficult for comprehensive integration of multimodal information via representation learning, and therefore pose a great challenge to further improvements of cancer survival prediction. Results To solve the above problems, we propose a novel method called cross-aligned multimodal representation learning (CAMR), which generates both modality-invariant and -specific representations for more accurate cancer survival prediction. Specifically, a cross-modality representation alignment learning network is introduced to reduce modality gaps by effectively learning modality-invariant representations in a common subspace, which is achieved by aligning the distributions of different modality representations through adversarial training. Besides, we adopt a cross-modality fusion module to fuse modality-invariant representations into a unified cross-modality representation for each patient. Meanwhile, CAMR learns modality-specific representations which complement modality-invariant representations and therefore provides a holistic view of the multimodal data for cancer survival prediction. Comprehensive experiment results demonstrate that CAMR can successfully narrow modality gaps and consistently yields better performance than other survival prediction methods using multimodal data. Availability CAMR is freely available at https://github.com/wxq-ustc/CAMR Supplementary information Supplementary data are available at Bioinformatics online.

show abstract

Section: Introductionmentioning

confidence: 99%

CAMR: cross-aligned multimodal representation learning for cancer survival prediction

Shi

Wang

et al. 2023

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…It has been applied in the medical field to tackle multimodality data to infer health or disease state of patients. For instance, Ning et al used representation learning to induce a latent shared space from multimodality medical data for Alzheimer's disease (AD) diagnosis and cancer prognosis analysis (Ning et al 2021a, Ning et al 2022. Zhou et al proposed a latent space inducing ensemble SVM classifier for the early dementia diagnosis with multi-modality neuroimaging data (Zhou et al 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Zheng et al proposed a modality-aware representation learning and graph learning based framework for disease prediction with multimodality (Zheng et al 2022). If solving the latent representation learning as an optimization problem (Zhou et al 2020, Ning et al 2021a, Ning et al 2022, it can not only provide an integrated architecture for simultaneous data fusion and dimension reduction, but also leverage data intrinsic properties, e.g. inter-/intramodality information and data-task correlation, in the optimization process by formulating them as regulation constrains.…”

Section: Introductionmentioning

confidence: 99%

Discriminative fusion of moments-aligned latent representation of multimodality medical data

Xie,

Zhong,

Yang

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Fusion of multimodal medical data provides multifaceted, disease-relevant information for diagnosis or prognosis prediction modeling. Traditional fusion strategies such as feature concatenation often fail to learn hidden complementary and discriminative manifestations from high-dimensional multimodal data. To this end, we proposed a methodology for the integration of multimodality medical data by matching their moments in a latent space, where the hidden, shared information of multimodal data is gradually learned by optimization with multiple feature collinearity and correlation constrains. We first obtained the multimodal hidden representations by learning mappings between the original domain and shared latent space. Within this shared space, we utilized several relational regularizations, including data attribute preservation, feature collinearity and feature-task correlation, to encourage learning of the underlying associations inherent in multimodal data. The fused multimodal latent features were finally fed to a logistic regression classifier for diagnostic prediction. Extensive evaluations on three independent clinical datasets have demonstrated the effectiveness of the proposed method in fusing multimodal data for medical prediction modeling.

show abstract

CoADS: Cross attention based dual-space graph network for survival prediction of lung cancer using whole slide images

Zhao

Hou

Teng

et al. 2023

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

Relation-Aware Shared Representation Learning for Cancer Prognosis Analysis With Auxiliary Clinical Variables and Incomplete Multi-Modality Data

Cited by 13 publications

References 46 publications

CAMR: cross-aligned multimodal representation learning for cancer survival prediction

CAMR: cross-aligned multimodal representation learning for cancer survival prediction

Discriminative fusion of moments-aligned latent representation of multimodality medical data

CoADS: Cross attention based dual-space graph network for survival prediction of lung cancer using whole slide images

Contact Info

Product

Resources

About