Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion

Abstract

Purpose We propose a deep learning‐based approach to breast mass classification in sonography and compare it with the assessment of four experienced radiologists employing breast imaging reporting and data system 4th edition lexicon and assessment protocol. Methods Several transfer learning techniques are employed to develop classifiers based on a set of 882 ultrasound images of breast masses. Additionally, we introduce the concept of a matching layer. The aim of this layer is to rescale pixel intensities of the grayscale ultrasound images and convert those images to red, green, blue (RGB) to more efficiently utilize the discriminative power of the convolutional neural network pretrained on the ImageNet dataset. We present how this conversion can be determined during fine‐tuning using back‐propagation. Next, we compare the performance of the transfer learning techniques with and without the color conversion. To show the usefulness of our approach, we additionally evaluate it using two publicly available datasets. Results Color conversion increased the areas under the receiver operating curve for each transfer learning method. For the better‐performing approach utilizing the fine‐tuning and the matching layer, the area under the curve was equal to 0.936 on a test set of 150 cases. The areas under the curves for the radiologists reading the same set of cases ranged from 0.806 to 0.882. In the case of the two separate datasets, utilizing the proposed approach we achieved areas under the curve of around 0.890. Conclusions The concept of the matching layer is generalizable and can be used to improve the overall performance of the transfer learning techniques using deep convolutional neural networks. When fully developed as a clinical tool, the methods proposed in this paper have the potential to help radiologists with breast mass classification in ultrasound.

Artificial intelligence in breast ultrasonography

“…These advantages may translate into ultrasound. More recently DL algorithms have been increasingly applied to breast ultrasound mostly with feasibility studies for automated detection, differential diagnosis, and segmentation [9][10][11][12][13][14][15]. Ultrasound has just begun in the development of a DL-based AI system, relative to mammography, and has unique characteristics over the developmental process.…”

A Temporal Sequence Dual-Branch Network for Classifying Hybrid Ultrasound Data of Breast Cancer

“…The network consists of two components to identify malignant tumors and recognize solid nodules in a cascade manner, which improve classification accuracy and sensitivity. Byra et al [25] presented a matching layer for utilize a pre-trained model on the dataset with 3-channel natural images in grayscale ultrasound images. So, the aim of this layer is to rescale pixel intensities of the grayscale ultrasound images and convert those images to red, green, blue (RGB).…”

Identification of Breast Malignancy by Marker-Controlled Watershed Transformation and Hybrid Feature Set for Healthcare

“…There are certain bottlenecks where conventional methods with hybrid features demonstrated the capability of a better solution [21]. Recently, deep learning and convolutional neural network (CNN)-based methodologies are employed for benign or malignant lesion recognition [22,23], but the cost of computation complexity of these methods are a major barriers in clinical applications [24,25]. To overcome this barrier, researchers have been considering various methods to lessen the time and cost related with deep learning application.…”