Feature Learning Using Stacked Autoencoder for Shared and Multimodal Fusion of Medical Images

Singh, Vikas; Verma, Nishchal K.; Islam, Zeeshan; Cui, Yan

doi:10.1007/978-981-13-1132-1_5

Cited by 4 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6 Structure of a bimodal AE across modalities. For example, the authors of [210][211][212] designed multimodal systems based on SAEs, where the encoder side of the architecture represents and compresses each unimodal feature separately, and the decoder side constructs the latent (shared) representation of the inputs in a unsupervised manner. Figure 6 shows the coupling mechanism of two separate AEs (bimodal AE) for both modalities (audio and video) into a jointly shared representation hierarchy where the encoder and decoder components are independent of each other.…”

Section: Shared Representationmentioning

confidence: 99%

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

et al. 2021

View full text Add to dashboard Cite

The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research.

show abstract

Section: Shared Representationmentioning

confidence: 99%

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The building block of deep learning models for multimodality fusion is its ability to extract relevant features for image fusion. Recently, researchers have implemented deep learning for biomedical imaging problems associated with MRI, CT, and PET images, respectively 83 . Kaur and Singh proposed a deep belief network for image fusion, where the proposed technique evaluates the fusion dataset acquired from the initial feature extraction procedure 84 .…”

Section: Ecs Transactions 107 (1) 3649-3673 (2022)mentioning

confidence: 99%

Multimodal Classification on PET/CT Image Fusion for Lung Cancer: A Comprehensive Survey

Das

Chandra

2022

ECS Trans.

View full text Add to dashboard Cite

Medical image fusion has become essential for accurate diagnosis. For example, a lung cancer diagnosis is currently conducted with the help of multimodality image fusion to find anatomical and functional information about the tumor and metabolic measurements to identify the lung cancer stage and metastatic information of the disease. Generally, the success of multimodality imaging for lung cancer diagnosis is due to the combination of PET and CT imaging advantages while minimizing their respective limitations. However, medical image fusion involves the registration of two different modalities, which is time-consuming and technically challenging, and it is a cause of concern in a clinical setting. Therefore, the paper's main objective is to identify the most efficient medical image fusion techniques and the recent advances by conducting a collective survey. In addition, the study delves into the impact of deep learning techniques for image fusion and their effectiveness in automating the image fusion procedure with better image quality while preserving essential clinical information.

show abstract

“…The stacked sparse auto-encoder [10] is a neural network that learns features from unlabeled data. It includes many layers of sparse auto-encoders where the outputs of each layer will be linked to the inputs of the following layer.…”

Section: Stacked Sparse Auto-encodersmentioning

confidence: 99%

Sparse Wavelet Auto-encoder for Covid-19 Cases Identification

Lazrag

Ali

Ejbali

2021

Hybrid Intelligent Systems

View full text Add to dashboard Cite

Diagnosis and understanding of disease progression require an interpretation of medical images, would take a lot of time in manual interpretation of the large amount of medical images accumulated. Thus, the automatic analysis and understanding of the medical images becomes an active research topic. In this case, feature extraction from the medical images plays an important role in obtaining diagnostic performance . In this context, we propose a Covid-19 cases identification based on sparse coding, wavelet analysis for feature extraction and AE for feature modeling. Our approach is based on sparse coding and wavelet analysis techniques for image representation and it is tested with the COVID-19 dataset. The experimental results demonstrate the performance of our system.

show abstract

Feature Learning Using Stacked Autoencoder for Shared and Multimodal Fusion of Medical Images

Cited by 4 publications

References 25 publications

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

Multimodal Classification on PET/CT Image Fusion for Lung Cancer: A Comprehensive Survey

Sparse Wavelet Auto-encoder for Covid-19 Cases Identification

Contact Info

Product

Resources

About