Video Polyp Segmentation: A Deep Learning Perspective

Abstract

We present the first comprehensive video polyp segmentation (VPS) study in the deep learning era. Over the years, developments in VPS are not moving forward with ease due to the lack of a large-scale dataset with fine-grained segmentation annotations. To address this issue, we first introduce a high-quality frame-by-frame annotated VPS dataset, named SUN-SEG, which contains 158 690 colonoscopy video frames from the well-known SUN-database. We provide additional annotation covering diverse types, i.e., attribute, object mask, boundary, scribble, and polygon. Second, we design a simple but efficient baseline, named PNS+, which consists of a global encoder, a local encoder, and normalized self-attention (NS) blocks. The global and local encoders receive an anchor frame and multiple successive frames to extract long-term and short-term spatial-temporal representations, which are then progressively refined by two NS blocks. Extensive experiments show that PNS+ achieves the best performance and real-time inference speed (170 fps), making it a promising solution for the VPS task. Third, we extensively evaluate 13 representative polyp/object segmentation models on our SUN-SEG dataset and provide attribute-based comparisons. Finally, we discuss several open issues and suggest possible research directions for the VPS community. Our project and dataset are publicly available at https://github.com/GewelsJI/VPS.

Research on Application of Big Data Technology in Cost Control of Medical Insurance For China

“…Hybrid 2/3D CNN framework [33] is used to aggregate spatio-temporal correlation and obtain better segmentation results. PNS+ [18] is the first study to comprehensively introduce the work related to video polyp segmentation in deep learning and the first to introduce a high-quality fine-grained annotated VPS dataset named SUN-SEG [30]. At the same time, a global encoder and a local encoder are designed in PNS+ to extract the long-term and short-term feature representation, respectively, and introduce a self-attention block to update the receptive field dynamically.…”

“…It contains 1,106 short video clips with a total of 158,690 frames, including 378 positive and 728 negative cases. We follow the same training/testing setting as in PNS+ [18] and only conduct experiments on positive cases. For training, we use 40% of the SUN-SEG dataset, including 112 clips with 19,544 frames.…”

“…Instead of relying on images, methods have been proposed to use colonoscopy videos fully. These methods are categorized as video polyp segmentation (VPS), where convolutional neural networks (CNNs) have been widely employed [33,15,18,44,43,20,55]. For instance, Puyal et al [33] proposed a hybrid VPS framework, where a 2D network acts as the backbone for extracting spatial features and a 3D network ensures temporal consistency.…”

“…For instance, Puyal et al [33] proposed a hybrid VPS framework, where a 2D network acts as the backbone for extracting spatial features and a 3D network ensures temporal consistency. Ji et al [18] comprehensively introduced the work related to video polyp segmentation in deep learning and the proposed model, PNS+, is the first to introduce a high-quality fine-grained annotated VPS dataset named SUN-SEG [30].…”

Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection

“…Therefore, camouflaged object detection (COD) presents a significantly more intricate challenge compared to traditional salient object detection (SOD) or other object segmentation. Recently, it has piqued ever-growing research interest from the computer vision community and facilitates many valuable real-life applications, such as search and rescue [1], species discovery [2], medical analysis (e.g., polyp segmentation [3], [4], [5], lung infection segmentation [6], and cell segmentation [7]), agricultural management [8], [9], and industrial defect detection [10].…”