The Importance of Skip Connections in Biomedical Image Segmentation

Drozdzal, Michal; Vorontsov, Eugene; Chartrand, Gabriel; Kadoury, Samuel; Pal, Chris

doi:10.1007/978-3-319-46976-8_19

Cited by 814 publications

(536 citation statements)

References 15 publications

Supporting

Mentioning

482

Contrasting

Unclassified

Order By: Relevance

“…U‐Net is an encoder–decoder network in which features on the encoder are transferred to the decoder layers through skip connections . The skip connection maintains the details of the prediction by concatenating the features of the encoder and the decoder . SegNet and DeconvNet also utilize features from low‐level layers of the encoder in a similar manner.…”

Section: Introductionmentioning

confidence: 99%

Endometrium segmentation on transvaginal ultrasound image using key‐point discriminator

et al. 2019

View full text Add to dashboard Cite

Purpose Transvaginal ultrasound imaging provides useful information for diagnosing endometrial pathologies and reproductive health. Endometrium segmentation in transvaginal ultrasound (TVUS) images is very challenging due to ambiguous boundaries and heterogeneous textures. In this study, we developed a new segmentation framework which provides robust segmentation against ambiguous boundaries and heterogeneous textures of TVUS images. Methods To achieve endometrium segmentation from TVUS images, we propose a new segmentation framework with a discriminator guided by four key points of the endometrium (namely, the endometrium cavity tip, the internal os of the cervix, and the two thickest points between the two basal layers on the anterior and posterior uterine walls). The key points of the endometrium are defined as meaningful points that are related to the characteristics of the endometrial morphology, namely the length and thickness of the endometrium. In the proposed segmentation framework, the key‐point discriminator distinguishes a predicted segmentation map from a ground‐truth segmentation map according to the key‐point maps. Meanwhile, the endometrium segmentation network predicts accurate segmentation results that the key‐point discriminator cannot discriminate. In this adversarial way, the key‐point information containing endometrial morphology characteristics is effectively incorporated in the segmentation network. The segmentation network can accurately find the segmentation boundary while the key‐point discriminator learns the shape distribution of the endometrium. Moreover, the endometrium segmentation can be robust to the heterogeneous texture of the endometrium. We conducted an experiment on a TVUS dataset that contained 3,372 sagittal TVUS images and the corresponding key points. The dataset was collected by three hospitals (Ewha Woman’s University School of Medicine, Asan Medical Center, and Yonsei University College of Medicine) with the approval of the three hospitals’ Institutional Review Board. For verification, fivefold cross‐validation was performed. Result The proposed key‐point discriminator improved the performance of the endometrium segmentation, achieving 82.67 % for the Dice coefficient and 70.46% for the Jaccard coefficient. In comparison, on the TVUS images UNet, showed 58.69 % for the Dice coefficient and 41.59 % for the Jaccard coefficient. The qualitative performance of the endometrium segmentation was also improved over the conventional deep learning segmentation networks. Our experimental results indicated robust segmentation by the proposed method on TVUS images with heterogeneous texture and unclear boundary. In addition, the effect of the key‐point discriminator was verified by an ablation study. Conclusion We proposed a key‐point discriminator to train a segmentation network for robust segmentation of the endometrium with TVUS images. By utilizing the key‐point information, the proposed method showed more reliable and accurate segmentation performance and outperformed the con...

show abstract

Section: Introductionmentioning

confidence: 99%

Endometrium segmentation on transvaginal ultrasound image using key‐point discriminator

et al. 2019

View full text Add to dashboard Cite

show abstract

“…As demonstrated in these studies, the skip connections designed in 3D‐FCNs or 3D‐UNets were very important to help recover the full spatial resolution at the network outputs, which is suitable for voxel‐wise segmentation tasks. Various typical extensions include the extended U‐Net based on the DenseNet, or the short skip connection . In this work, we utilized a coupled skip connection between the two 3D‐UNets for CT or PET, taking advantage of both modalities to produce two separate segmentations.…”

Section: Discussionmentioning

confidence: 99%

“…Various typical extensions include the extended U-Net based on the DenseNet, 55 or the short skip connection. 56 In this work, we utilized a coupled skip connection between the two 3D-UNets for CT or PET, taking advantage of both modalities to produce two separate segmentations. Although our proposed method achieved good results, designing a more efficient feature fusion architecture would be very beneficial.…”

Section: D Limitationsmentioning

confidence: 99%

Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

et al. 2019

View full text Add to dashboard Cite

Purpose To investigate the use and efficiency of 3‐D deep learning, fully convolutional networks (DFCN) for simultaneous tumor cosegmentation on dual‐modality nonsmall cell lung cancer (NSCLC) and positron emission tomography (PET)‐computed tomography (CT) images. Methods We used DFCN cosegmentation for NSCLC tumors in PET‐CT images, considering both the CT and PET information. The proposed DFCN‐based cosegmentation method consists of two coupled three‐dimensional (3D)‐UNets with an encoder‐decoder architecture, which can communicate with the other in order to share complementary information between PET and CT. The weighted average sensitivity and positive predictive values denoted as Scores, dice similarity coefficients (DSCs), and the average symmetric surface distances were used to assess the performance of the proposed approach on 60 pairs of PET/CTs. A Simultaneous Truth and Performance Level Estimation Algorithm (STAPLE) of 3 expert physicians’ delineations were used as a reference. The proposed DFCN framework was compared to 3 graph‐based cosegmentation methods. Results Strong agreement was observed when using the STAPLE references for the proposed DFCN cosegmentation on the PET‐CT images. The average DSCs on CT and PET are 0.861 ± 0.037 and 0.828 ± 0.087, respectively, using DFCN, compared to 0.638 ± 0.165 and 0.643 ± 0.141, respectively, when using the graph‐based cosegmentation method. The proposed DFCN cosegmentation using both PET and CT also outperforms the deep learning method using either PET or CT alone. Conclusions The proposed DFCN cosegmentation is able to outperform existing graph‐based segmentation methods. The proposed DFCN cosegmentation shows promise for further integration with quantitative multimodality imaging tools in clinical trials.

show abstract

“…was applied for the segmentation of tumors, which required a longer processing time due to the depth of the network. In general, using CNN for semantic and medical image segmentation has achieved better results than those using traditional segmentation approaches …”

Section: Introductionmentioning

confidence: 99%

“…The U-Net developed by Dong et al 8 was applied for the segmentation of tumors, which required a longer processing time due to the depth of the network. In general, using CNN for semantic [9][10][11][12] and medical image 7,8,[13][14][15][16][17][18][19] segmentation has achieved better results than those using traditional segmentation approaches. [2][3][4][5] Ibragimov et al 20 first proposed deep learning-based algorithms for OAR segmentation of head and neck CT images in 2017.…”

Section: Introductionmentioning

confidence: 99%

Accurate and rapid CT image segmentation of the eyes and surrounding organs for precise radiotherapy

et al. 2019

View full text Add to dashboard Cite

Objective The precise segmentation of organs at risk (OARs) is of importance for improving therapeutic outcomes and reducing injuries of patients undergoing radiotherapy. In this study, we developed a new approach for accurate computed tomography (CT) image segmentation of the eyes and surrounding organs, which is first locating then segmentation (FLTS). Methods The FLTS approach was composed of two steps: (a) classification of CT images using convolutional neural networks (CNN), and (b) segmentation of the eyes and surrounding organs using modified U‐shape networks. In order to obtain optimal performance, we enhanced our training datasets by random jitter and rotation. Results This model was trained and verified using the clinical datasets that were delineated by experienced physicians. The dice similarity coefficient (DSC) was employed to evaluate the performance of our segmentation method. The average DSCs for the segmentation of the pituitary, left eye, right eye, left eye lens, right eye lens, left optic nerve, and right optic nerve were 90%, 94%, 93.5%, 84.5%, 84.3%, 80.3%, and 82.2%, respectively. Conclusion We developed a new network‐based approach for rapid and accurate CT image segmentation of the eyes and surrounding organs. This method is accurate and efficient, and is suitable for clinical use.

show abstract

The Importance of Skip Connections in Biomedical Image Segmentation

Cited by 814 publications

References 15 publications

Endometrium segmentation on transvaginal ultrasound image using key‐point discriminator

Endometrium segmentation on transvaginal ultrasound image using key‐point discriminator

Simultaneous cosegmentation of tumors in PET‐CT images using deep fully convolutional networks

Accurate and rapid CT image segmentation of the eyes and surrounding organs for precise radiotherapy

Contact Info

Product

Resources

About