Semi-MsST-GAN: A Semi-Supervised Segmentation Method for Corneal Ulcer Segmentation in Slit-Lamp Images

Abstract

Corneal ulcer is a common leading cause of corneal blindness. It is difficult to accurately segment corneal ulcers due to the following problems: large differences in the pathological shapes between point-flaky and flaky corneal ulcers, blurred boundary, noise interference, and the lack of sufficient slit-lamp images with ground truth. To address these problems, in this paper, we proposed a novel semi-supervised multi-scale self-transformer generative adversarial network (Semi-MsST-GAN) that can leverage unlabeled images to improve the performance of corneal ulcer segmentation in fluorescein staining of slit-lamp images. Firstly, to improve the performance of segmenting the corneal ulcer regions with complex pathological features, we proposed a novel multi-scale self-transformer network (MsSTNet) as the MsST-GAN generator, which can guide the model to aggregate the low-level weak semantic features with the high-level strong semantic information and adaptively learn the spatial correlation in feature maps. Then, to further improve the segmentation performance by leveraging unlabeled data, the semi-supervised approach based on the proposed MsST-GAN was explored to solve the problem of the lack of slit-lamp images with corresponding ground truth. The proposed Semi-MsST-GAN was comprehensively evaluated on the public SUSTech-SYSU dataset, which contains 354 labeled and 358 unlabeled fluorescein staining slit-lamp images. The results showed that, compared with other state-of-the-art methods, our proposed method achieves better performance with comparable efficiency.

Two‐stage coarse‐to‐fine method for pathological images in medical decision‐making systems

“…The network uses a superpixel segmentation method based on the Bounded Asymmetric Gaussian Mixture Model and a Multiview Attention Module to improve the precision of segmentation of cell nuclei with small areas and fuzzy boundaries. Wang et al [40] designed and proposed a multi-transformer GAN to segment corneal ulcer images. The network uses a multi-scale self-transformer module to segment the ulcer region, capturing global, long-range pixel dependencies using different levels of multi-scale features and making full use of unlabelled samples.…”

“…Wang et al. [40] designed and proposed a multi‐transformer GAN to segment corneal ulcer images. The network uses a multi‐scale self‐transformer module to segment the ulcer region, capturing global, long‐range pixel dependencies using different levels of multi‐scale features and making full use of unlabelled samples.…”

A transformer-based generative adversarial network for brain tumor segmentation

“…Various methods (Isola et al, 2017 ; Han et al, 2018 ; Xue et al, 2018 ; Choi et al, 2019 ; Dong et al, 2019 ; Oh et al, 2020 ; Ding et al, 2021 ; He et al, 2021 ; Nishio et al, 2021 ; Wang T. et al, 2021 ; Zhan et al, 2021 ; Asis-Cruz et al, 2022 ) were proposed to explore the possibility of GAN in medical image segmentation. Xue et al ( 2018 ) used U-Net as the generator and proposed a multi-scale L 1 loss to minimize the distance of the feature maps of predictions and masks for the medical image segmentation of brain tumors.…”

Automated Detection of Corneal Ulcer Using Combination Image Processing and Deep Learning

“…Again, they evaluated their model using the SUSTech-SYSU dataset and achieved better segmentation performance than the state-of-the-art CNN-based methods. However, the limited number of slit lamp images available for training and evaluation represents a limitation for both studies [ 12 ].…”