Precise Yet Efficient Semantic Calibration and Refinement in ConvNets for Real-time Polyp Segmentation from Colonoscopy Videos

Abstract

We propose a novel convolutional neural network (ConvNet) equipped with two new semantic calibration and refinement approaches for automatic polyp segmentation from colonoscopy videos. While ConvNets set state-of-the-are performance for this task, it is still difficult to achieve satisfactory results in a real-time manner, which is a necessity in clinical practice. The main obstacle is the huge semantic gap between high-level features and low-level features, making it difficult to take full advantage of complementary semantic information contained in these hierarchical features. Compared with existing solutions, which either directly aggregate these features without considering the semantic gap or employ sophisticated non-local modeling techniques to refine semantic information by introduce many extra computational costs, the proposed ConvNet is able to more precisely yet efficiently calibrate and refine semantic information for better segmentation performance without increasing model complexity; we call the proposed ConvNet as SCR-Net, which has two key modules. We first propose a semantic calibration module (SCM) to effectively transmit the semantic information from high-level layers to low-level layers by learning the semantic-spatial relations during the training procedure. We then propose a semantic refinement module (SRM) to, based on the features calibrated by SCM, enhance the discrimination capability of the features for targeting objects. Extensive experiments on the Kvasir-SEG dataset demonstrate that the proposed SCR-Net is capable of achieving better segmentation accuracy than state-of-the-art approaches with a faster speed. The proposed techniques are general enough to be applied to similar applications where precise and efficient multi-level feature fusion is critical. The code is available at https://github.com/jiafuz/SCR-Net.

Semantic Segmentation of Digestive Abnormalities from WCE Images by Using AttResU-Net Architecture

“…As shown in Table 5 AttResU-Net outperformed the existing studies conducted on the WCE red lesion dataset in terms of accuracy with a difference of 10.16%. Table 6 compares the suggested approaches’ performance to that of [ 30 , 43 ]. These existing networks are based on deep learning techniques such as ResUNet, PSP-Net, Attention-UNet, and CE-Net.…”

“…The framework achieved 94.42% global accuracy when tested on a publicly available dataset. The paper [ 30 ] presents a method for precise and efficient semantic calibration and refinement in convolutional neural networks (ConvNets) for real-time polyp segmentation from colonoscopy videos. The goal is to improve the accuracy of polyp segmentation, which is an important task in the diagnosis and monitoring of colon cancer.…”

Parallel matters: Efficient polyp segmentation with parallel structured feature augmentation modules

“…Early deep learning‐based polyp segmentation methods [37, 38] use fully convolutional networks (FCN), while, later on, U‐Net [39] based methods become popular, such as SFA [40], PolypSeg [41], ACS [42] and SCR‐Net [43]. These methods either adopt U‐Net directly or introduce U‐Net enhanced architectures for improved performances.…”

“…They do not need big datasets as the deep learning techniques, but may suffer from low accuracy due to their limited representation capabilities. Early deep learning-based polyp segmentation methods [37,38] use fully convolutional networks (FCN), while, later on, U-Net [39] based methods become popular, such as SFA [40], PolypSeg [41], ACS [42] and SCR-Net [43]. These methods either adopt U-Net directly or introduce U-Net enhanced architectures for improved performances.…”

Video Polyp Segmentation: A Deep Learning Perspective

“…Inspired by the vast success of UNet [34] in biomedical image segmentation, UNet++ [35] and ResUNet [36] were employed for polyp segmentation for improved performance. Furthermore, PolypSeg [37] , ACS [38] , ColonSegNet [39] , and SCR-Net [40] explore the effectiveness of UNet-enhanced architecture on adaptively learning semantic contexts. As the newly-proposed methods, SANet [41] and MSNet [42] design the shallow attention module and subtraction unit, respectively, to achieve precise and efficient segmentation.…”