MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation

Abstract

Methods based on convolutional neural networks have improved the performance of biomedical image segmentation. However, most of these methods cannot efficiently segment objects of variable sizes and train on small and biased datasets, which are common for biomedical use cases. While methods exist that incorporate multi-scale fusion approaches to address the challenges arising with variable sizes, they usually use complex models that are more suitable for general semantic segmentation problems. In this paper, we propose a novel architecture called Multi-Scale Residual Fusion Network (MSRF-Net), which is specially designed for medical image segmentation. The proposed MSRF-Net is able to exchange multi-scale features of varying receptive fields using a Dual-Scale Dense Fusion (DSDF) block. Our DSDF block can exchange information rigorously across two different resolution scales, and our MSRF sub-network uses multiple DSDF blocks in sequence to perform multi-scale fusion. This allows the preservation of resolution, improved information flow and propagation of both high-and low-level features to obtain accurate segmentation maps. The proposed MSRF-Net allows to capture object variabilities and provides improved results on different biomedical datasets. Extensive experiments on MSRF-Net demonstrate that the proposed method outperforms the cutting-edge medical image segmentation methods on four publicly available datasets. We achieve the Dice Coefficient (DSC) of 0.9217, 0.9420, and 0.9224, 0.8824 on Kvasir-SEG, CVC-ClinicDB, 2018 Data Science Bowl dataset, and ISIC-2018 skin lesion segmentation challenge dataset respectively. We further conducted generalizability tests and achieved DSC of 0.7921 and 0.7575 on CVC-ClinicDB and Kvasir-SEG, respectively.

Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge

“…Later, Zhang et al 8 proposed a hybrid method combining both transformer-based network and CNN to capture global dependencies and the low-level spatial features for the segmentation task. Inspired by high-resolution network 51 , Srivastava et al 10 proposed multi-scale residual fusion network (MSRF-Net) that allows information exchange across multiple scales and showed improved generalisability on unseen datasets. All of these encoder-decoder architectures were evaluated only on still images.…”

“…Most deep learning-based detection [3][4][5] and segmentation [6][7][8][9] methods are trained and tested on the same center dataset and WLE modality only. These supervised deep learning techniques has a major issue in not being able to generalise to an unseen data from a different center population 10 or even different modality from the same center 11 . Also, the type of endoscope used also adds to the compromise in robustness.…”

FCN-Transformer Feature Fusion for Polyp Segmentation

“…To improve on the segmentation of polyps in colonoscopy images, a range of deep learning (DL) -based solutions [8,13,14,17,19,22,28,30,32,37] have been proposed. Such solutions are designed to automatically predict segmentation maps for colonoscopy images, in order to provide assistance to clinicians performing colonoscopy procedures.…”

“…Such solutions are designed to automatically predict segmentation maps for colonoscopy images, in order to provide assistance to clinicians performing colonoscopy procedures. These solutions have traditionally used fully convolutional networks (FCNs) [1,9,10,[13][14][15]17,25,28,39]. However, transformerbased architectures [24,[32][33][34]36] have recently become popular for semantic segmentation and shown superior performance over FCN-based alternatives.…”

Semantic Segmentation Using Pixel-Wise Adaptive Label Smoothing via Self-Knowledge Distillation for Limited Labeling Data

“…The goal of semantic segmentation is to predict the predefined class (or label) of each pixel, which is fundamental yet challenging in computer vision. Owing to its increasing importance, it is widely adopted in various applications using vision sensors, such as autonomous driving [ 1 , 2 ], 3D reconstruction [ 3 ], and medical image analysis [ 4 , 5 ]. In recent years, deep convolutional neural networks (DCNNs) have achieved significant performance improvements and have been the dominant solution for semantic segmentation.…”