Abdomen CT multi‐organ segmentation using token‐based MLP‐Mixer

Abstract

Background: Manual contouring is very labor-intensive, time-consuming, and subject to intra-and inter-observer variability. An automated deep learning approach to fast and accurate contouring and segmentation is desirable during radiotherapy treatment planning. Purpose: This work investigates an efficient deep-learning-based segmentation algorithm in abdomen computed tomography (CT) to facilitate radiation treatment planning. Methods: In this work, we propose a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) and convolutional neural network (CNN) for multi-organ segmentation in abdomen CT images. The proposed model has a similar structure to V-net, while a proposed MLP-Convolutional block replaces each convolutional block. The MLP-Convolutional block consists of three components: an early convolutional block for local features extraction and feature resampling, a token-based MLP-Mixer layer for capturing global features with high efficiency, and a token projector for pixel-level detail recovery. We evaluate our proposed network using: (1) an institutional dataset with 60 patient cases and (2) a public dataset (BCTV) with 30 patient cases. The network performance was quantitatively evaluated in three domains: (1) volume similarity between the ground truth contours and the network predictions using the Dice score coefficient (DSC), sensitivity, and precision; (2) surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS); and (3) the computational complexity reported by the number of network parameters, training time, and inference time. The performance of the proposed network is compared with other state-of -the-art networks. Results: In the institutional dataset, the proposed network achieved the following volume similarity measures when averaged over all organs: DSC = 0.912, sensitivity = 0.917, precision = 0.917, average surface similarities were HD = 11.95 mm, MSD = 1.90 mm, RMS = 3.86 mm. The proposed network achieved DSC = 0.786 and HD = 9.04 mm on the public dataset. The network also shows statistically significant improvement, which is evaluated by a two-tailed Wilcoxon Mann-Whitney U test, on right lung (MSD where the maximum p-value is 0.001), spinal cord (sensitivity, precision, HD, RMSD where p-value ranges from 0.001 to 0.039), and stomach (DSC where the maximum p-value is 0.01) over all other competing networks. On the public dataset, the network report statistically significant improvement, which is shown by the Wilcoxon Mann-Whitney test, on pancreas (HD where the maximum p-value is 0.006), left (HD where

A vision-GNN framework for retinopathy classification using optical coherence tomography

“…Rapid improvements in artificial intelligence (AI) have enabled broad application of AI-assisted medical image analysis (classification, segmentation, registration, synthesis) pipelines [4][5][6][7][8][9][10][11][12][13], including semi-automated retinopathy detection systems using machine learning (ML) classifiers [14] based on human-designed features as well as fully-automated deep learning (DL) systems [15,16]. Currently, mainstream DL frameworks include Multilayer Perceptrons (MLP), Transformers [17], and Convolutional Neural Networks (CNN) [18], which can only take in grid or sequence data.…”

Prostate gleason score prediction via MRI using capsule network

“…Radiomics, the extraction of image features, has been used to build models aiding in lesion detection, lesion synthesis and cancer prognosis [9,10]. In recent years, many deep learning models have achieved state-of-the-art results in image diagnosis [11][12][13], organ segmentation [14,15] and treatment planning [16,17] in medicine. Recent advances in deep learning have also demonstrated success of convolutional neural networks (CNN) to detect cancer from MRI.…”

Ultrasound-based dominant intraprostatic lesion classification with Swin Transformer

“…Due to its feasibility in detecting DILs, B-mode ultrasound could be potentially integrated into the intraoperative treatment planning for a focal boost [2]. With the development of computer-aided diagnosis (CAD), powerful artificial intelligence (AI) tools such as deep learning (DL) and reinforcement learning algorithms have been applied in medical image analysis [3][4][5][6][7][8][9][10], such as image segmentation [11], image registration [12] and image synthesis [13]. With deeper layers, DL methods can adaptively extracts feature maps at multiple resolution levels, and demonstrates strength in computer vision [14].…”