Deep Learning to Obtain Simultaneous Image and Segmentation Outputs From a Single Input of Raw Ultrasound Channel Data

Abstract

Single plane wave transmissions are promising for automated imaging tasks requiring high ultrasound frame rates over an extended field of view. However, a single plane wave insonification typically produces suboptimal image quality. To address this limitation, we are exploring the use of deep neural networks (DNNs) as an alternative to delay-and-sum beamforming. The objectives of this work are to obtain information directly from raw channel data and to simultaneously generate both a segmentation map for automated ultrasound tasks and a corresponding ultrasound B-mode image for interpretable supervision of the automation. We focus on visualizing and segmenting anechoic targets surrounded by tissue and ignoring or de-emphasizing less important surrounding structures. DNNs trained with Field II simulations were tested with simulated, experimental phantom, and in vivo datasets that were not included during training. With unfocused input channel data (i.e., prior to the application of receive time delays), simulated, experimental phantom, and in vivo test datasets achieved mean ± standard deviation Dice similarity coefficients of 0.92 ± 0.13, 0.92±0.03, and 0.77±0.07, respectively, and generalized contrastto-noise ratios (gCNR) of 0.95±0.08, 0.93±0.08, and 0.75±0.14, respectively. With subaperture beamformed channel data and a modification to the input layer of the DNN architecture to accept these data, the fidelity of image reconstruction increased (e.g., mean gCNR of multiple acquisitions of two in vivo breast cysts ranged 0.89-0.96), but DNN display frame rates were reduced from 395 Hz to 287 Hz. Overall, the DNNs successfully translated feature representations learned from simulated data to phantom and in vivo data, which is promising for this novel approach to simultaneous ultrasound image formation and segmentation.

Deep-fUS: Functional ultrasound imaging of the brain using deep learning and sparse data

“…Deep learning is steadily revolutionizing modern medicine and the medical imaging field as a whole 36 , and medical ultrasound represents no exception. In the last few years, deep CNNs have been the object of increasing attention in anatomical ultrasound image reconstruction, with a particular emphasis on models that aim to restore high-end image quality while reducing data sampling, transmission, and processing 25,26, 28,37 . With the exception of a single study using deep learning of color Doppler images 38 , however, CNNs have not been applied as extensively to blood flow imaging, and to the best of our knowledge this is the first attempt to implement an end-to-end network to perform power Doppler reconstruction from sparse ultrasonic datasets (Fig.…”

“…Deep learning and CNNs are drawing increasing attention for the reconstruction and processing of biomedical images with sparse data [35][36][37] . In medical ultrasound, several strategies have been proposed to restore high image quality while reducing data sampling, transmission, and processing 30,31,33,38 . With the exception of a single preliminary study reporting deep learning of color Doppler images 39 , however, CNNs have not been applied as extensively to ultrasound imaging of blood flows.…”

“…Inspired by recent advances in the use of convolutional neural networks (CNNs) for the reconstruction and processing of biomedical images from sparse data [18][19][20] , in this paper we propose Deep-fUS, an end-to-end deep learning approach to reconstruct power Doppler images from sub-optimally sampled compound datasets. Recently, CNN solutions have been proposed in medical ultrasound imaging for applications including contrast improvement 21 and image de-speckling 22 , ultrasound contrast agent localization and tracking 23,24 , and for undersampled and adaptive beamforming [25][26][27][28] . Here we implement a deep CNN based on the U-Net architecture 29 that learns a reconstruction mapping between the sequence of compound ultrasound data and the power Doppler output image (Figure 1g-h).…”

“…Variants of this model have been used for biomedical image reconstruction in applications spanning compressed sensing MRI 21 , sparse-projection photoacoustic imaging 22,23 , and sparse X-ray computed tomography 24,25 . Prior CNN applications in medical ultrasound imaging include contrast improvement 26 and image de-speckling 27 , ultrasound contrast agent localization and tracking 28,29 , and under-sampled and adaptive beamforming [30][31][32][33] . Our network learns a reconstruction mapping between the sparse sequence of compound ultrasound data and the power Doppler output image, without requiring any prior model-based knowledge (Fig.…”

Deep Learning for Ultrasound Image Formation: CUBDL Evaluation Framework and Open Datasets

“…The CUBDL organizers then downloaded the submitted models and launched a Python script to perform evaluation [15] on the internationally crowd-sourced database of test data [16]. Evaluation metrics advertised since the launch of the CUBDL website [14] were pre-selected by the CUBDL organizers based on literature from multiple groups reporting beamforming with deep learning (e.g., [8], [9], [11], [23]) and based on common computer vision literature containing assessments of network complexity (e.g., [24]- [26]).…”