FMRNet: A fused network of multiple tumoral regions for breast tumor classification with ultrasound images

Abstract

Purpose Recent studies have illustrated that the peritumoral regions of medical images have value for clinical diagnosis. However, the existing approaches using peritumoral regions mainly focus on the diagnostic capability of the single region and ignore the advantages of effectively fusing the intratumoral and peritumoral regions. In addition, these methods need accurate segmentation masks in the testing stage, which are tedious and inconvenient in clinical applications. To address these issues, we construct a deep convolutional neural network that can adaptively fuse the information of multiple tumoral‐regions (FMRNet) for breast tumor classification using ultrasound (US) images without segmentation masks in the testing stage. Methods To sufficiently excavate the potential relationship, we design a fused network and two independent modules to extract and fuse features of multiple regions simultaneously. First, we introduce two enhanced combined‐tumoral (EC) region modules, aiming to enhance the combined‐tumoral features gradually. Then, we further design a three‐branch module for extracting and fusing the features of intratumoral, peritumoral, and combined‐tumoral regions, denoted as the intratumoral, peritumoral, and combined‐tumoral module. Especially, we design a novel fusion module by introducing a channel attention module to adaptively fuse the features of three regions. The model is evaluated on two public datasets including UDIAT and BUSI with breast tumor ultrasound images. Two independent groups of experiments are performed on two respective datasets using the fivefold stratified cross‐validation strategy. Finally, we conduct ablation experiments on two datasets, in which BUSI is used as the training set and UDIAT is used as the testing set. Results We conduct detailed ablation experiments about the proposed two modules and comparative experiments with other existing representative methods. The experimental results show that the proposed method yields state‐of‐the‐art performance on both two datasets. Especially, in the UDIAT dataset, the proposed FMRNet achieves a high accuracy of 0.945 and a specificity of 0.945, respectively. Moreover, the precision (PRE = 0.909) even dramatically improves by 21.6% on the BUSI dataset compared with the existing method of the best result. Conclusion The proposed FMRNet shows good performance in breast tumor classification with US images, and proves its capability of exploiting and fusing the information of multiple tumoral‐regions. Furthermore, the FMRNet has potential value in classifying other types of cancers using multiple tumoral‐regions of other kinds of medical images.

Breast Cancer Diagnosis Using Iterative Fourier-Bessel Decomposition Method Based CNN-kernel Features

“…So, the fused features obtained from both sub-band images may be the reason for good classification performance. The proposed framework has not used any segmentation techniques and does not even require ground truth of segmented tumor which was used in other state-of-art methods [6]- [9]. This makes the proposed work more automated and easily implementable.…”

“…The fused features have been extracted from two types of modules like the enhanced combined tumoral module and the Intra tumoral, peritumoral, and combined tumoral (IPC) module, which are then used for classification purposes. In [9], authors used UNet to segment lesion areas from sonographic images and the independent component analysis (ICA) method was used on segmented images for extracting features. The obtained features are then fused with deep automated features.…”

HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images

“…Qian et al [4] aggregated the multimodal ultrasound images for an explainable prediction to support the clinical decision-making of the sonographers and increase the confidence levels of the decision. Cui et al [26] proposed an FMRNet to fuse combined tumoral, intratumoral and peritumoral regions to represent the whole tumor heterogeneous. Di et al [27] introduced a saliency-guided approach to differentiate the foreground and background regions by two separated branches.…”

A VGG attention vision transformer network for benign and malignant classification of breast ultrasound images

“…28 In 2021, Cui et al constructed a deep CNN that can fully exploit the features of multiple regions and their potential relationships with categories. 29 These previous CNNs had achieved success in BUS classification. They could extract local features well but were not good at extracting global features; also, the reception field of CNN was still limited.…”