Combining convolutional neural networks and self-attention for fundus diseases identification

Abstract

Early detection of lesions is of great significance for treating fundus diseases. Fundus photography is an effective and convenient screening technique by which common fundus diseases can be detected. In this study, we use color fundus images to distinguish among multiple fundus diseases. Existing research on fundus disease classification has achieved some success through deep learning techniques, but there is still much room for improvement in model evaluation metrics using only deep convolutional neural network (CNN) architectures with limited global modeling ability; the simultaneous diagnosis of multiple fundus diseases still faces great challenges. Therefore, given that the self-attention (SA) model with a global receptive field may have robust global-level feature modeling ability, we propose a multistage fundus image classification model MBSaNet which combines CNN and SA mechanism. The convolution block extracts the local information of the fundus image, and the SA module further captures the complex relationships between different spatial positions, thereby directly detecting one or more fundus diseases in retinal fundus image. In the initial stage of feature extraction, we propose a multiscale feature fusion stem, which uses convolutional kernels of different scales to extract low-level features of the input image and fuse them to improve recognition accuracy. The training and testing were performed based on the ODIR-5k dataset. The experimental results show that MBSaNet achieves state-of-the-art performance with fewer parameters. The wide range of diseases and different fundus image collection conditions confirmed the applicability of MBSaNet.

Sq-Swin: Siamese Quadratic Swin Transformer for Lettuce Browning Prediction

“…Therefore, some recent works have started to incorporate Self-Attention (SA) [23]- [25] into CNN architectures for diagnosis to add global attention rather than just local attention. Particularly, Wang et al [25] propose a multi-level fundus image classification model MBSaNet that combines CNN and Self-Attention mechanisms.…”

“…Therefore, some recent works have started to incorporate Self-Attention (SA) [23]- [25] into CNN architectures for diagnosis to add global attention rather than just local attention. Particularly, Wang et al [25] propose a multi-level fundus image classification model MBSaNet that combines CNN and Self-Attention mechanisms. The convolutional block extracts local information of the fundus image, and the SA module further captures the complex relationship between different spatial positions, thereby directly detecting one or more fundus diseases in the fundus image.…”

Predictor Model for Chronic Kidney Disease using Adaptive Gradient Clipping with Deep Neural Nets

“…Transformers were proposed in [21] for machine translation and much recently they are deployed in image processing applications also. Multiple works try combining CNN-like architectures with self-attention [22] [23] [24], some replacing the convolutions entirely [25] [26]. But in largescale image recognition, classic ResNetlike architectures are still state of the art [27] [28] [29] [30].…”

SMMF: a self-attention-based multi-parametric MRI feature fusion framework for the diagnosis of bladder cancer grading

“…Wang et al. ( 26 ) introduced a multi-stage fundus image classification model, which combines CNN and an attention mechanism to enhance the accuracy of fundus disease recognition.…”