Multi-modal learning with missing data for cancer diagnosis using histopathological and genomic data

Cui, Can; Asad, Zuhayr; Dean, William F.; Smith, Isabelle T.; Madden, Christopher; Bao, Shunxin; Landman, Bennett A.; Roland, Joseph T.; Coburn, Lori A.; Wilson, Keith T.; Zwerner, Jeffrey P.; Zhao, Shilin; Wheless, Lee; Huo, Yuankai

doi:10.1117/12.2612318

Cited by 1 publication

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al (2019a) proposed cross partial multi-modal networks (CPM-Net), which directly optimizes latent representations to reconstruct modality-incomplete data, regardless of the unavailable patterns. Cui et al (2022b) applied CPM-Net to the task of glioma grading and achieved good performance. While these methods can handle incomplete cases, they do not explicitly utilize consistent-complementary information among different modalities, which may limit their performance.…”

Section: Incomplete Multimodal Fusion Of Histology and Genomicsmentioning

confidence: 99%

“…Another strategy aims to create a shared latent feature space that remains robust even when some modalities are unavailable. These approaches project or compress individual unimodal features into a latent feature space by using various constraints or alternate updates (Cheerla and Gevaert 2019, Zhang et al 2019a, Vale Silva and Rohr 2020, Cui et al 2022a, 2022b This shared knowledge representation can be applied to downstream tasks as a multimodal fusion feature and performs well even when certain modalities are missing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dual-space disentangled-multimodal network (DDM-net) for glioma diagnosis and prognosis with incomplete pathology and genomic data

Qiu,

Zhao,

Zhao

et al. 2024

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective. Effective fusion of histology slides and molecular profiles from genomic data has shown great potential in the diagnosis and prognosis of gliomas. However, it remains challenging to explicitly utilize the consistent-complementary information among different modalities and create comprehensive representations of patients. Additionally, existing researches mainly focus on complete multi-modality data and usually fail to construct robust models for incomplete samples. Approach. In this paper, we propose a dual-space disentangled-multimodal network (DDM-net) for glioma diagnosis and prognosis. DDM-net disentangles the latent features generated by two separate variational autoencoders (VAEs) into common and specific components through a dual-space disentangled approach, facilitating the construction of comprehensive representations of patients. More importantly, DDM-net imputes the unavailable modality in the latent feature space, making it robust to incomplete samples. Main results. We evaluated our approach on the TCGA-GBMLGG dataset for glioma grading and survival analysis tasks. Experimental results demonstrate that the proposed method achieves superior performance compared to state-of-the-art methods, with a competitive AUC of 0.952 and a C-index of 0.768. Significance. The proposed model may help the clinical understanding of gliomas and can serve as an effective fusion model with multimodal data. Additionally, it is capable of handling incomplete samples, making it less constrained by clinical limitations.

show abstract

Section: Incomplete Multimodal Fusion Of Histology and Genomicsmentioning

confidence: 99%