BSDA-Net: A Boundary Shape and Distance Aware Joint Learning Framework for Segmenting and Classifying OCTA Images

Lin, Li; Wang, Zhonghua; Wu, Jiewei; Huang, Yijin; Lyu, Junyan; Cheng, Pujin; Wu, Jiong; Tang, Xiaoying

doi:10.1007/978-3-030-87237-3_7

Cited by 33 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They utilized spatial and channel attention modules to improve segmentation performance. Lin et al [43] introduced a joint learning method for FAZ segmentation and diagnostic classification. They used the detected FAZ to improve the performance of diagnostic classification networks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Retinal Structure Detection in OCTA Image via Voting-Based Multitask Learning

Hao

Shen

Zhu

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Automated detection of retinal structures, such as retinal vessels (RV), the foveal avascular zone (FAZ), and retinal vascular junctions (RVJ), are of great importance for understanding diseases of the eye and clinical decision-making. In this paper, we propose a novel Votingbased Adaptive Feature Fusion multi-task network (VAFF-Net) for joint segmentation, detection, and classification of RV, FAZ, and RVJ in optical coherence tomography angiography (OCTA). A task-specific voting gate module is proposed to adaptively extract and fuse different features for specific tasks at two levels: features at different spatial positions from a single encoder, and features from multiple encoders. In particular, since the complexity of the microvasculature in OCTA images makes simultaneous precise localization and classification of retinal vascular junctions into bifurcation/crossing a challenging task, we specifically design a task head by combining the heatmap regression and grid classification. We take advantage of three different en face angiograms from various retinal layers, rather than following existing methods that use only a single en face. To validate the superiority of our VAFF-Net, we carry out extensive experiments on three OCTA datasets acquired using different imaging devices, and the results demonstrate that the proposed method performs on the whole better than either the state-of-the-art singlepurpose methods or existing multi-task learning solutions.We also demonstrate that our multi-task learning method generalizes across other imaging modalities, such as color fundus photography, and may potentially be used as a general multi-task learning tool. For the first time in the field of retinal OCTA image analysis, we construct three datasets for multiple structure detection. To facilitate further research, part of these datasets with the source code and evaluation benchmark have been released for public access.

show abstract

Section: Related Workmentioning

confidence: 99%

“…All existing automated methods for extracting the multiple retina structures use the en face image of the IVC only [17], [18], [43]. We stated in Sec.…”

Section: B Effectiveness Of Multiple En Face Inputsmentioning

confidence: 99%

Retinal Structure Detection in OCTA Image via Voting-Based Multitask Learning

Hao

Shen

Zhu

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…Retinal vessels are observable in retinal fundus images and optical coherence tomography angiography (OCTA) images, while corneal nerve fibers are identifiable in confocal corneal microscopy (CCM) images. It has been suggested that early signs of many ophthalmic diseases are reflected by microvascular and capillary abnormalities [5], [6]. Collectively, accurate segmentation of various curvilinear structures is of great importance for computer-aided diagnosis, quantitative analysis and early screening, especially in ophthalmology.…”

Section: Introductionmentioning

confidence: 99%

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

Li¹,

Peng²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Weakly-supervised learning (WSL) has been proposed to alleviate the conflict between data annotation cost and model performance through employing sparsely-grained (i.e., point-, box-, scribble-wise) supervision and has shown promising performance, particularly in the image segmentation field. However, it is still a very challenging problem due to the limited supervision, especially when only a small number of labeled samples are available. Additionally, almost all existing WSL segmentation methods are designed for star-convex structures which are very different from curvilinear structures such as vessels and nerves. In this paper, we propose a novel sparsely annotated segmentation framework for curvilinear structures, named YoloCurvSeg, based on image synthesis. A background generator delivers image backgrounds that closely match real distributions through inpainting dilated skeletons. The extracted backgrounds are then combined with randomly emulated curves generated by a Space Colonization Algorithm-based foreground generator and through a multilayer patch-wise contrastive learning synthesizer. In this way, a synthetic dataset with both images and curve segmentation labels is obtained, at the cost of only one or a few noisy skeleton annotations.</p> <p>Finally, a segmenter is trained with the generated dataset and possibly an unlabeled dataset. The proposed YoloCurvSeg is evaluated on four publicly available datasets (OCTA500, CORN, DRIVE and CHASEDB1) and the results show that YoloCurvSeg outperforms state-of-the-art WSL segmentation methods by large margins. With only one noisy skeleton annotation (respectively 0.14%, 0.03%, 1.40%, and 0.65% of the full annotation), YoloCurvSeg achieves more than 97% of the fully-supervised performance on each dataset. Code and datasets will be released at https://github.com/llmir/YoloCurvSeg.</p>

show abstract

“…classification [19] and structure segmentation [14,22], benefiting from supervision of large-scale labeled datasets [27]. However, manual delineation is timeconsuming and labor-intensive, especially for large-scale datasets.…”

Section: Introductionmentioning

confidence: 99%

Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification

Cai¹,

Li²,

He³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

A large-scale labeled dataset is a key factor for the success of supervised deep learning in computer vision. However, a limited number of annotated data is very common, especially in ophthalmic image analysis, since manual annotation is time-consuming and labor-intensive. Selfsupervised learning (SSL) methods bring huge opportunities for better utilizing unlabeled data, as they do not need massive annotations. With an attempt to use as many as possible unlabeled ophthalmic images, it is necessary to break the dimension barrier, simultaneously making use of both 2D and 3D images. In this paper, we propose a universal self-supervised Transformer framework, named Uni4Eye, to discover the inherent image property and capture domain-specific feature embedding in ophthalmic images. Uni4Eye can serve as a global feature extractor, which builds its basis on a Masked Image Modeling task with a Vision Transformer (ViT) architecture. We employ a Unified Patch Embedding module to replace the origin patch embedding module in ViT for jointly processing both 2D and 3D input images. Besides, we design a dual-branch multitask decoder module to simultaneously perform two reconstruction tasks on the input image and its gradient map, delivering discriminative representations for better convergence. We evaluate the performance of our pre-trained Uni4Eye encoder by fine-tuning it on six downstream ophthalmic image classification tasks. The superiority of Uni4Eye is successfully established through comparisons to other state-of-the-art SSL pre-training methods.

show abstract

BSDA-Net: A Boundary Shape and Distance Aware Joint Learning Framework for Segmenting and Classifying OCTA Images

Cited by 33 publications

References 27 publications

Retinal Structure Detection in OCTA Image via Voting-Based Multitask Learning

Retinal Structure Detection in OCTA Image via Voting-Based Multitask Learning

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification

Contact Info

Product

Resources

About