Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Abstract

Medical image analysis using deep learning has recently been prevalent, showing great performance for various downstream tasks including medical image segmentation and its sibling, volumetric image segmentation. Particularly, a typical volumetric segmentation network strongly relies on a voxel grid representation which treats volumetric data as a stack of individual voxel 'slices', which allows learning to segment a voxel grid to be as straightforward as extending existing image-based segmentation networks to the 3D domain. However, using a voxel grid representation requires a large memory footprint, expensive test-time and limiting the scalability of the solutions. In this paper, we propose Point-Unet, a novel method that incorporates the efficiency of deep learning with 3D point clouds into volumetric segmentation. Our key idea is to first predict the regions of interest in the volume by learning an attentional probability map, which is then used for sampling the volume into a sparse point cloud that is subsequently segmented using a point-based neural network. We have conducted the experiments on the medical volumetric segmentation task with both a small-scale dataset Pancreas and large-scale datasets BraTS18, BraTS19, and BraTS20 challenges. A comprehensive benchmark on different metrics has shown that our context-aware Point-Unet robustly outperforms the SOTA voxel-based networks at both accuracies, memory usage during training, and time consumption during testing. Our code is available at https://github.com/VinAIResearch/Point-Unet.

A Comprehensive Study on Medical Image Segmentation using Deep Neural Networks

“…or 3D medical image segmentation, CNN models often employ a 3D kernel to extract spatial features or utilize GNN or Transformer. Some of SOTA models in this field include 3D U-Net [40], V-Net [39], nnU-Net [84], HighRes3dNet [85], 3D-Res-Unet [86], DenseVNet [87], UNETR [88], SegResNet [89], Point-Unet [90], and others.…”

3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation

“…Since the introduction of UNet [1], [2], UNet-based CNNs have achieved impressive performance in various modalities of MIS, e.g. brain tumor [3], [4], [5], infant brain [6], [7], liver tumor [8], optic disc [9], retina [10], lung [11], and cell [12], etc. However, each feature map in such encoder-decoder architecture only contains information about the presence of the feature, and the network relies on fixed learned weight matrix to link features between layers.…”