Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R‐CNN

Abstract

Purpose Automatic breast ultrasound (ABUS) imaging has become an essential tool in breast cancer diagnosis since it provides complementary information to other imaging modalities. Lesion segmentation on ABUS is a prerequisite step of breast cancer computer‐aided diagnosis (CAD). This work aims to develop a deep learning‐based method for breast tumor segmentation using three‐dimensional (3D) ABUS automatically. Methods For breast tumor segmentation in ABUS, we developed a Mask scoring region‐based convolutional neural network (R‐CNN) that consists of five subnetworks, that is, a backbone, a regional proposal network, a region convolutional neural network head, a mask head, and a mask score head. A network block building direct correlation between mask quality and region class was integrated into a Mask scoring R‐CNN based framework for the segmentation of new ABUS images with ambiguous regions of interest (ROIs). For segmentation accuracy evaluation, we retrospectively investigated 70 patients with breast tumor confirmed with needle biopsy and manually delineated on ABUS, of which 40 were used for fivefold cross‐validation and 30 were used for hold‐out test. The comparison between the automatic breast tumor segmentations and the manual contours was quantified by I) six metrics including Dice similarity coefficient (DSC), Jaccard index, 95% Hausdorff distance (HD95), mean surface distance (MSD), residual mean square distance (RMSD), and center of mass distance (CMD); II) Pearson correlation analysis and Bland–Altman analysis. Results The mean (median) DSC was 85% ± 10.4% (89.4%) and 82.1% ± 14.5% (85.6%) for cross‐validation and hold‐out test, respectively. The corresponding HD95, MSD, RMSD, and CMD of the two tests was 1.646 ± 1.191 and 1.665 ± 1.129 mm, 0.489 ± 0.406 and 0.475 ± 0.371 mm, 0.755 ± 0.755 and 0.751 ± 0.508 mm, and 0.672 ± 0.612 and 0.665 ± 0.729 mm. The mean volumetric difference (mean and ± 1.96 standard deviation) was 0.47 cc ([−0.77, 1.71)) for the cross‐validation and 0.23 cc ([−0.23 0.69]) for hold‐out test, respectively. Conclusion We developed a novel Mask scoring R‐CNN approach for the automated segmentation of the breast tumor in ABUS images and demonstrated its accuracy for breast tumor segmentation. Our learning‐based method can potentially assist the clinical CAD of breast cancer using 3D ABUS imaging.

Automatic semantic segmentation of breast tumors in ultrasound images based on combining fuzzy logic and deep learning—A feasibility study

“…Artificial intelligence (AI) including DL has emerged recently though various applications in healthcare [7][8][9]. Efficient cancer characterization in BUS images can be obtained by appropriate automatic SS scheme [10][11][12][13][14][15][16]. Efficient DL based automatic SS, being a challenging task, is aiming to label each pixel in an image with a corresponding class using supervised learning [11,[17][18][19][20][21][22].…”

“…Y. Lei et al [ 15 ], have introduced their study for breast tumor segmentation in three dimensional (3D) ABUS, proposing a developed Mask scoring region-based CNN (Mask R-CNN) consists of five subnetworks: a backbone, a regional proposal network, a region CNN head, a mask head, and a mask score head. Their approach has been validated on 70 patients’ images with ground truth manual contour, resulting in an efficient segmentation of breast cancer’s volume from ABUS images.…”

Echocardiographic image multi‐structure segmentation using Cardiac‐SegNet

“…A potential way is to enlarge the number of potential bounding boxes, and use weighted bounding boxes for segmentation consolidation, as recommended in Mask R-CNN 17 and Mask scoring R-CNN. 21 In contrast, we only estimated one detected ROI for one structure. By considering more potential bounding boxes, the failure of segmentation or misclassification may be diminished via consolidation.…”

“…In this work, the detected ROI was represented for each structure by its center , class , and bounding box . In contrast, the existing Mask R‐CNN method uses multiple anchors (potential ROIs) centered at many possible locations and then regresses the bounding box offsets to classify these anchors 17,21 . However, this method requires substantial computational memory due to the large number of anchor candidates needed for the highly variable organ locations and shapes.…”

“…In contrast, the existing Mask R-CNN method uses multiple anchors (potential ROIs) centered at many possible locations and then regresses the bounding box offsets to classify these anchors. 17,21 However, this method requires substantial computational memory due to the large number of anchor candidates needed for the highly variable organ locations and shapes. These ambiguous anchors would increase the difficulty of training and thus affect the detection performance.…”