Visual ensemble selection of deep convolutional neural networks for 3D segmentation of breast tumors on dynamic contrast enhanced MRI

Abstract

Objectives To develop a visual ensemble selection of deep convolutional neural networks (CNN) for 3D segmentation of breast tumors using T1-weighted dynamic contrast-enhanced (T1-DCE) MRI. Methods Multi-center 3D T1-DCE MRI (n = 141) were acquired for a cohort of patients diagnosed with locally advanced or aggressive breast cancer. Tumor lesions of 111 scans were equally divided between two radiologists and segmented for training. The additional 30 scans were segmented independently by both radiologists for testing. Three 3D U-Net models were trained using either post-contrast images or a combination of post-contrast and subtraction images fused at either the image or the feature level. Segmentation accuracy was evaluated quantitatively using the Dice similarity coefficient (DSC) and the Hausdorff distance (HD95) and scored qualitatively by a radiologist as excellent, useful, helpful, or unacceptable. Based on this score, a visual ensemble approach selecting the best segmentation among these three models was proposed. Results The mean and standard deviation of DSC and HD95 between the two radiologists were equal to 77.8 ± 10.0% and 5.2 ± 5.9 mm. Using the visual ensemble selection, a DSC and HD95 equal to 78.1 ± 16.2% and 14.1 ± 40.8 mm was reached. The qualitative assessment was excellent (resp. excellent or useful) in 50% (resp. 77%). Conclusion Using subtraction images in addition to post-contrast images provided complementary information for 3D segmentation of breast lesions by CNN. A visual ensemble selection allowing the radiologist to select the most optimal segmentation obtained by the three 3D U-Net models achieved comparable results to inter-radiologist agreement, yielding 77% segmented volumes considered excellent or useful. Key Points • Deep convolutional neural networks were developed using T1-weighted post-contrast and subtraction MRI to perform automated 3D segmentation of breast tumors. • A visual ensemble selection allowing the radiologist to choose the best segmentation among the three 3D U-Net models outperformed each of the three models. • The visual ensemble selection provided clinically useful segmentations in 77% of cases, potentially allowing for a valuable reduction of the manual 3D segmentation workload for the radiologist and greatly facilitating quantitative studies on non-invasive biomarker in breast MRI.

3D Breast Cancer Segmentation in DCE‐MRI Using Deep Learning With Weak Annotation

“…26,32 There have also been attempts at more complex forms of breast cancer segmentation, including one study that complemented suspicious parts of a public dataset that were not previously annotated, 29 and another that trained a model on only locally advanced breast cancers and achieved high accuracy. 25,33 Marked BPE did not affect the quantitative analysis but was a major factor in the qualitative analysis. This difference may be due to the minimal volume of the BPE, which appears as an enhancement focus compared to the tumor itself.…”

“…25,26 Rahimpour et al presented how to obtain more accurate segmentation results through a visual ensemble after obtaining various segmentation results. 25 Yue et al implemented an interactive labeling workflow, manually segmenting 10% of the training dataset and training the rest of data based on it. 26 The trained deep learning model performed well in 3D breast cancer segmentation, with a median Dice similarity coefficient of 0.75 for whole breast and 0.89 for ROI.…”

“…Although the number of steps increased compared to simple manual segmentation, this process allowed us to select and focus on cases, making it a more efficient use of time and human workload. There have been other attempts to reduce the burden of building an annotated dataset 25,26 . Rahimpour et al presented how to obtain more accurate segmentation results through a visual ensemble after obtaining various segmentation results 25 .…”

“…There have been other attempts to reduce the burden of building an annotated dataset 25,26 . Rahimpour et al presented how to obtain more accurate segmentation results through a visual ensemble after obtaining various segmentation results 25 . Yue et al implemented an interactive labeling workflow, manually segmenting 10% of the training dataset and training the rest of data based on it 26 …”

Editorial for “3D Breast Cancer Segmentation in DCE‐MRI Using Deep Learning With Weak Annotation”

“…The performance of the UNETR model was in accordance with the DSC values reported by other researchers employing alternative segmentation algorithms. [6][7][8] Regarding the qualitative analysis of the segmentation results, the segmentation was successfully done in 83% of the cases derived from inputs 1 and 2, and from these, 95% were considered as acceptable detection. The authors also evaluated the performance of the segmentation according to baseline characteristics and found significant differences for the whole breast and main lesion.…”