Multimodal learning using convolution neural network and Sparse Autoencoder

Vu, Tien Duong; Yang, Hyung-Jeong; Nguyen, Van Du; Oh, Aran; Kim, Mi-Sun

doi:10.1109/bigcomp.2017.7881683

Cited by 46 publications

(39 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study obtained an AD/CN classification accuracy of 92.87% and an MCI conversion prediction accuracy of 72.44%. Vu et al (2017) applied SAE and 3D CNN to subjects with MRI and FDG PET scans to yield an AD/CN classification accuracy of 91.1%. Liu et al (2018a) decomposed 3D PET images into a sequence of 2D slices and used a combination of 2D CNN and RNNs to learn the intra-slice and inter-slice features for classification, respectively.…”

Section: Resultsmentioning

confidence: 99%

Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data

Nho

Saykin

2019

Front. Aging Neurosci.

472

240

View full text Add to dashboard Cite

Deep learning, a state-of-the-art machine learning approach, has shown outstanding performance over traditional machine learning in identifying intricate structures in complex high-dimensional data, especially in the domain of computer vision. The application of deep learning to early detection and automated classification of Alzheimer's disease (AD) has recently gained considerable attention, as rapid progress in neuroimaging techniques has generated large-scale multimodal neuroimaging data. A systematic review of publications using deep learning approaches and neuroimaging data for diagnostic classification of AD was performed. A PubMed and Google Scholar search was used to identify deep learning papers on AD published between January 2013 and July 2018. These papers were reviewed, evaluated, and classified by algorithm and neuroimaging type, and the findings were summarized. Of 16 studies meeting full inclusion criteria, 4 used a combination of deep learning and traditional machine learning approaches, and 12 used only deep learning approaches. The combination of traditional machine learning for classification and stacked auto-encoder (SAE) for feature selection produced accuracies of up to 98.8% for AD classification and 83.7% for prediction of conversion from mild cognitive impairment (MCI), a prodromal stage of AD, to AD. Deep learning approaches, such as convolutional neural network (CNN) or recurrent neural network (RNN), that use neuroimaging data without pre-processing for feature selection have yielded accuracies of up to 96.0% for AD classification and 84.2% for MCI conversion prediction. The best classification performance was obtained when multimodal neuroimaging and fluid biomarkers were combined. Deep learning approaches continue to improve in performance and appear to hold promise for diagnostic classification of AD using multimodal neuroimaging data. AD research that uses deep learning is still evolving, improving performance by incorporating additional hybrid data types, such as—omics data, increasing transparency with explainable approaches that add knowledge of specific disease-related features and mechanisms.

show abstract

Section: Resultsmentioning

confidence: 99%

Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data

Nho

Saykin

2019

Front. Aging Neurosci.

472

240

View full text Add to dashboard Cite

show abstract

“…This should be expected because the authors operate on FDG-PET, not the amyloid PET scans used in this work. Furthermore, the accuracy of the models in [24] is noticeably lower. Namely, the "Simple CNN" model, which is analogous to the models used in this work, performs at 80.62% and 81.93% for MRI and FDG-PET, respectively.…”

Section: Table I: Classification Accuraciesmentioning

confidence: 95%

Neuroimaging modality fusion in Alzheimer’s classification using convolutional neural networks

et al. 2019

View full text Add to dashboard Cite

Automated methods for Alzheimer’s disease (AD) classification have the potential for great clinical benefits and may provide insight for combating the disease. Machine learning, and more specifically deep neural networks, have been shown to have great efficacy in this domain. These algorithms often use neurological imaging data such as MRI and FDG PET, but a comprehensive and balanced comparison of the MRI and amyloid PET modalities has not been performed. In order to accurately determine the relative strength of each imaging variant, this work performs a comparison study in the context of Alzheimer’s dementia classification using the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset with identical neural network architectures. Furthermore, this work analyzes the benefits of using both modalities in a fusion setting and discusses how these data types may be leveraged in future AD studies using deep learning.

show abstract

“…This is also a direction for future research in this field. Vu et al [61] proposed a fusion method based on the combination of sparse self-encoder and convolutional neural network. The preprocessing SAE was added to the CNN classifier, which is better than the previous CNN.…”

Section: Image Fusion Based On Deepmentioning

confidence: 99%

A Review of Multimodal Medical Image Fusion Techniques

Huang

Yang

Yin

et al. 2020

Computational and Mathematical Methods in Medicine

121

View full text Add to dashboard Cite

The medical image fusion is the process of coalescing multiple images from multiple imaging modalities to obtain a fused image with a large amount of information for increasing the clinical applicability of medical images. In this paper, we attempt to give an overview of multimodal medical image fusion methods, putting emphasis on the most recent advances in the domain based on (1) the current fusion methods, including based on deep learning, (2) imaging modalities of medical image fusion, and (3) performance analysis of medical image fusion on mainly data set. Finally, the conclusion of this paper is that the current multimodal medical image fusion research results are more significant and the development trend is on the rise but with many challenges in the research field.

show abstract

Multimodal learning using convolution neural network and Sparse Autoencoder

Cited by 46 publications

References 10 publications

Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data

Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data

Neuroimaging modality fusion in Alzheimer’s classification using convolutional neural networks

A Review of Multimodal Medical Image Fusion Techniques

Contact Info

Product

Resources

About