Learning Two-Stream CNN for Multi-Modal Age-Related Macular Degeneration Categorization

Wang, Weisen; Li, Xirong; Xu, Zhiyan; Wu, Yu; Zhao, Jianchun; Ding, Dayong; Chen, Youxin

doi:10.1109/jbhi.2022.3171523

Cited by 18 publications

(16 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by pixtopixHD [5] , we design a CGAN to generate more training data and the overall architecture of our network is shown in Figure 2. We first trained a classification network with a single branch of fundus color photographs to get CAM images with pathological data [6] . Together with the original images, they are fed into the CGAN network as data pairs.…”

Section: Structure Of the Proposed Cgan Networkmentioning

confidence: 99%

Multi-modality network based on CGAN and attention mechanism for glaucoma grading

Liu¹,

Peng²,

Xiang

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

Glaucoma is a progressive optic neuropathy characterized by changes in the structure of the optic nerve head and visual field，which is one of the major irreversible blinding eye diseases worldwide. Early screening and timely diagnosis of glaucoma is of significant importance. Fundus color photography and optical coherence tomography (OCT) are the two most effective imaging modalities for glaucoma screening, where significant ocular structural changes, such as vertical cup-to-disc ratio (vCDR) on fundus images and retinal nerve fiber layer (RNFL) thickness on OCT volumes, can be present with both imaging modalities. In recent years, multi-modal deep learning methods have shown great advantages in image classification and segmentation tasks. In this paper, we propose a multi-modal glaucoma grading network with two main contributions: (1) To address the inherent shortage of multi-modal training data, conditional generative adversarial network (CGAN) is used to generate more synthetic images, extending the dataset over the only available dataset. (2) A multi-modality cross-attention (MMCA) module is proposed to further improve the classification accuracy.

show abstract

Section: Structure Of the Proposed Cgan Networkmentioning

confidence: 99%

Multi-modality network based on CGAN and attention mechanism for glaucoma grading

Liu¹,

Peng²,

Xiang

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

show abstract

“…The view of lesions provided by the different data helps ophthalmologists to build a comprehensive perception of the disease situation [ 13 – 16 ]. For example, OCT and CFP are two commonly-applied [ 17 ], as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

“…The CFP image provides a 2D projection of the retina with wide field of view, and the OCT captures a 3D cross-sectional view. Several automatic diagnosis algorithms using multi-modal images as input have emerged [ 14 , 16 , 18 ], including early input concatenation [ 18 ] or late features fusion for classification [ 14 , 16 ]. Hua et al [ 18 ] proposed a network to predict the severity of diabetic retinopathy (DR) based on the concatenation of the CFP images and swept-source optical coherence tomography angiography images.…”

Section: Introductionmentioning

confidence: 99%

“…Hua et al [ 18 ] proposed a network to predict the severity of diabetic retinopathy (DR) based on the concatenation of the CFP images and swept-source optical coherence tomography angiography images. Wang et al [ 16 ] extracted features CFP and OCT based on two different branches and then classified using their late-fused features. Expanding on the work [ 16 ], He et al [ 14 ] applied attention mechanisms to different branches of their model, which is opt to emphasize modality-specific features.…”

Section: Introductionmentioning

confidence: 99%

“…Wang et al [ 16 ] extracted features CFP and OCT based on two different branches and then classified using their late-fused features. Expanding on the work [ 16 ], He et al [ 14 ] applied attention mechanisms to different branches of their model, which is opt to emphasize modality-specific features. Thus, these algorithms tend to treat images of different modalities as independent images, but in fact there is a correspondence between images of the same patient in different modalities.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Cross-modal attention network for retinal disease classification based on multi-modal images

Liu,

Hu,

Qiu

et al. 2024

Biomed. Opt. Express

View full text Add to dashboard Cite

Multi-modal eye disease screening improves diagnostic accuracy by providing lesion information from different sources. However, existing multi-modal automatic diagnosis methods tend to focus on the specificity of modalities and ignore the spatial correlation of images. This paper proposes a novel cross-modal retinal disease diagnosis network (CRD-Net) that digs out the relevant features from modal images aided for multiple retinal disease diagnosis. Specifically, our model introduces a cross-modal attention (CMA) module to query and adaptively pay attention to the relevant features of the lesion in the different modal images. In addition, we also propose multiple loss functions to fuse features with modality correlation and train a multi-modal retinal image classification network to achieve a more accurate diagnosis. Experimental evaluation on three publicly available datasets shows that our CRD-Net outperforms existing single-modal and multi-modal methods, demonstrating its superior performance.

show abstract

Reliable Multimodality Eye Disease Screening via Mixture of Student’s t Distributions

Zou,

Lin,

Yuan

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning Two-Stream CNN for Multi-Modal Age-Related Macular Degeneration Categorization

Cited by 18 publications

References 36 publications

Multi-modality network based on CGAN and attention mechanism for glaucoma grading

Multi-modality network based on CGAN and attention mechanism for glaucoma grading

Cross-modal attention network for retinal disease classification based on multi-modal images

Reliable Multimodality Eye Disease Screening via Mixture of Student’s t Distributions

Contact Info

Product

Resources

About