Breast ultrasound image segmentation: A coarse‐to‐fine fusion convolutional neural network

Abstract

Purpose Breast ultrasound (BUS) image segmentation plays a crucial role in computer‐aided diagnosis systems for BUS examination, which are useful for improved accuracy of breast cancer diagnosis. However, such performance remains a challenging task owing to the poor image quality and large variations in the sizes, shapes, and locations of breast lesions. In this paper, we propose a new convolutional neural network with coarse‐to‐fine feature fusion to address the aforementioned challenges. Methods The proposed fusion network consists of an encoder path, a decoder path, and a core fusion stream path (FSP). The encoder path is used to capture the context information, and the decoder path is used for localization prediction. The FSP is designed to generate beneficial aggregate feature representations (i.e., various‐sized lesion features, aggregated coarse‐to‐fine information, and high‐resolution edge characteristics) from the encoder and decoder paths, which are eventually used for accurate breast lesion segmentation. To better retain the boundary information and alleviate the effect of image noise, we input the superpixel image along with the original image to the fusion network. Furthermore, a weighted‐balanced loss function was designed to address the problem of lesion regions having different sizes. We then conducted exhaustive experiments on three public BUS datasets to evaluate the proposed network. Results The proposed method outperformed state‐of‐the‐art (SOTA) segmentation methods on the three public BUS datasets, with average dice similarity coefficients of 84.71(±1.07), 83.76(±0.83), and 86.52(±1.52), average intersection‐over‐union values of 76.34(±1.50), 75.70(±0.98), and 77.86(±2.07), average sensitivities of 86.66(±1.82), 85.21(±1.98), and 87.21(±2.51), average specificities of 97.92(±0.46), 98.57(±0.19), and 99.42(±0.21), and average accuracies of 95.89(±0.57), 97.17(±0.3), and 98.51(±0.3). Conclusions The proposed fusion network could effectively segment lesions from BUS images, thereby presenting a new feature fusion strategy to handle challenging task of segmentation, while outperforming the SOTA segmentation methods. The code is publicly available at https://github.com/mniwk/CF2-NET.

Breast Ultrasound Image Segmentation Algorithm Using Adaptive Region Growing and Variation Level Sets

“…When compared to the level set model for multiclass [31], this study optimizes Jaccard; the value is 0.9999 in malignant and 0.9954 in benign, both of which are better than 0.9658. At the same time, the runtime is shorter, 0.233 s and 0.185 s in this study, which is shorter than 2.6 s. Considering the Dice, by comparing soft and hard attention multitask learning [38] and Coarse-to-Fine Fusion CNN [37], the scores are 0.9999 and 0.9977, respectively, both of which are better than 0.8142 and 0.8652.…”

“…rough the investigation of Dice and Jaccard, Jaccard and running time were compared to level set model for multiclass [31], and Dice was compared to Coarse-to-Fine Fusion CNN [37] and soft and hard attention multitask learning [38], respectively. By testing the surface of the public data set, the value of the method explained in this paper is better than that listed in level set model for multiclass [31] on Jaccard, which reduces the running time; when compared to the Dice index, the value of the Dice method in this paper is higher than that mentioned in Coarse-to-Fine Fusion CNN [37] and soft and hard attention multitask learning [38].…”

“…Huang et al [36] proposed a new segmentation method by dense prediction and local fusion of superpixels for breast anatomy with scarce labeled data and achieved good results. Wang et al [37] proposed a novel CNN with coarse-to-fine feature fusion to address the poor quality of breast ultrasound images and large variations in breast lesion size, shape, and location. Zhang et al [38] proposed a multitask learning model based on a soft and hard attention mechanism for the simultaneous segmentation and binary classification of breast ultrasound images, and the experimental results demonstrated the effectiveness of the model.…”

“…In this study, based on the segmentation algorithm combining RG and adaptive level set, the boundary points of the ARG are defined through the level set of the pixel neighborhood, so that the contour perimeter and area of the lesion region could be effectively identified. e Jaccard of Koshki et al [31] is 0.9658, and the run time is 2.6 seconds, whereas the Dice of Wang et al [37] is 0.8142, and that of Zhang et al [38] is 0.8152. Our work is to calculate the value of Jaccard and Dice used in an ARG algorithm for level set optimization, and the results exhibited better Jaccard and Dice.…”

Breast Tumor Ultrasound Image Segmentation Method Based on Improved Residual U-Net Network

“…With the development and application of computer technology, researchers choose to continuously train the tumor ultrasound image dataset to learn with the help of deep learning network and independently extract the information features in the dataset, so as to realize noninvasive tumor medical auxiliary judgment [ 7 ]. However, it should also be noted that, due to the deep structure of the depth network model, the gradient is easy to disappear when training the dataset to learn, which makes it difficult for the image segmentation network model to realize accurate and effective tumor region recognition.…”