A Multimodal Affinity Fusion Network for Predicting the Survival of Breast Cancer Patients

Abstract

Accurate survival prediction of breast cancer holds significant meaning for improving patient care. Approaches using multiple heterogeneous modalities such as gene expression, copy number alteration, and clinical data have showed significant advantages over those with only one modality for patient survival prediction. However, existing survival prediction methods tend to ignore the structured information between patients and multimodal data. We propose a multimodal data fusion model based on a novel multimodal affinity fusion network (MAFN) for survival prediction of breast cancer by integrating gene expression, copy number alteration, and clinical data. First, a stack-based shallow self-attention network is utilized to guide the amplification of tiny lesion regions on the original data, which locates and enhances the survival-related features. Then, an affinity fusion module is proposed to map the structured information between patients and multimodal data. The module endows the network with a stronger fusion feature representation and discrimination capability. Finally, the fusion feature embedding and a specific feature embedding from a triple modal network are fused to make the classification of long-term survival or short-term survival for each patient. As expected, the evaluation results on comprehensive performance indicate that MAFN achieves better predictive performance than existing methods. Additionally, our method can be extended to the survival prediction of other cancer diseases, providing a new strategy for other diseases prognosis.

A multimodal stacked ensemble model for breast cancer prognosis prediction: a proposed CNN hyperparameter optimization using the Tug of War (TWO) algorithm

“…CNA and gene expression data had unknown and null values during preprocessing-the weighted nearest neighbor algorithm [47] was used to remove and normalize these unwanted data points [5]. The gene expression features were further discretized and classified as underexpressed (-1), baseline (0), or overexpressed (1). With five discrete values, the CNA features remain unchanged (2, 1, 0, 1, 2).…”

“…However, the heterogeneity of these data has led to complexity because of the significant differences in dimensionality and data types. The process of integrating these data types into a single predictive model is complex [1]- [3]. A prognosis estimates the probability of risk, such as complications or death, occurring over a given period.…”

“…Predicting breast cancer prognosis has been challenging due to the lack of fast and effective methods in recent decades [1]- [3], [8]. Advanced optimization techniques such as metaheuristic algorithms for deep learning models enhance the accuracy and efficiency of breast cancer prognosis.…”

“…Multimodal learning provides a solution to the single modality challenges by combining and integrating complementary data from multiple sources to make a final decision. [1]- [3], [8], [14]- [16], [17]- [19].…”

Unsupervised Learning Based on Multiple Descriptors for WSIs Diagnosis

“…Literature studies have used supervised or weakly supervised learning methods or proposed fused models for a more accurate diagnosis of WSIs. The fused models [ 35 , 36 ] have demonstrated better computational performance than the specific set of optimal feature extractors, but these models are more dependent on the selected set of features or classifiers, which might not work with relatively rare types of cancer cases, extracted from WSI data. In contrast, our study helps to obtain sufficient information from WSIs, by using any number of independent multiple descriptors for unsupervised learning of features, and later, different classifiers (supervised) can be used to achieve high performance for any type of WSI classification.…”