Joint Spine Segmentation and Noise Removal From Ultrasound Volume Projection Images With Selective Feature Sharing

Abstract

Volume Projection Imaging from ultrasound data is a promising technique to visualize spine features and diagnose Adolescent Idiopathic Scoliosis. In this paper, we present a novel multi-task framework to reduce the scan noise in volume projection images and to segment different spine features simultaneously, which provides an appealing alternative for intelligent scoliosis assessment in clinical applications. Our proposed framework consists of two streams: i) A noise removal stream based on generative adversarial networks, which aims to achieve effective scan noise removal in a weakly-supervised manner, i.e., without paired noisy-clean samples for learning; ii) A spine segmentation stream, which aims to predict accurate bone masks. To establish the interaction between these two tasks, we propose a selective feature-sharing strategy to transfer only the beneficial features, while filtering out the useless or harmful information. We evaluate our proposed framework on both scan noise removal and spine segmentation tasks. The experimental results demonstrate that our proposed method achieves promising performance on both tasks, which provides an appealing approach to facilitating clinical diagnosis.

Automatic Segmentation of Abdominal Aortic Aneurysms from Time-Resolved 3D Ultrasound Images Using Deep Learning

“…Data augmentation is typically used in deep learning to increase the variety of the training dataset, and recent studies have shown the benefits of data augmentation for deep learning-based segmentation [10]. Therefore, these data augmentations, such as application of Gaussian noise, elastic transformation, rotation, and flipping have been used in stateof-the-art studies on ultrasound segmentation [11], [12], [13], [14]. However, due to the large volume of 3D+t US images available in this study and the fact that these augmentations lead to less realistic ultrasound images, data augmentation might not be beneficial, and models without augmentation were trained as well.…”

Is denoising necessary for ultrasound image segmentation deep learning: review and benchmark

“…For the sake of brevity, we denote the references for corresponding topics in the form of numbers in the bracket. Deep learning based segmentation model: segmentation building blocks (1) [19], [20], RNNs (2) [21], GNNs (3) [22], attention (4,5) [23], [24], Transformer (6,7) [25], [26], multi-scale (8,9) [27], [28], boundary correction block (10,11) [6], [29]; architecture, Encoder-Decoder (12) [1], detection based segmentation architecture (13)(14)(15) [19], [27], [30]- [35], generative models (16) [36]. Loss function: segmentation task oriented (17,18) [37], [38]; generative models oriented (16) [36]; supervision strategies oriented (19,20) [39], [40]; transfer learning oriented (21)(22)(23)(24) [41]-…”

“…Xie et al [58] applied the main task of denoising to preserve retinal structural information, where auxiliary segmentation task provided retina-related region information. Huang et al [5] applied multi-task denoising-segmentation for segmentation purpose, where the scan noise -generated from moving 2D scanning, 3D formation and anatomical plane projectionis significantly different from the non-Gaussian statistics of speckle noise.…”

“…This benchmarking has concluded that denoising pre-processing brings unstable and slight (if exists) performance improvement to the downstream task of deep learning ultrasound segmentation, and we recommend to regard it as a kind of deep learning hyper-parameter, which should be checked in clinical application to judge its real effect on segmentation. While multi-task denoising-segmentation [5] actually results in a segmentation performance degradation, which may be the limitation of the cross-task gap that is possibly generated from the different context reasoning and input-output workflows in generalization or domain transfer.…”

“…To tackle the problem of speckle noise, some traditional [50]- [52] and deep learning segmentation methods [53]- [57] have considered denoising or despeckling as a useful preprocessing, where denoised data that differs from the original input data in the boundaries and details, might lead to better segmentation feature representation. Moreover, some works applied multi-task deep learning denoisingsegmentation to fully utilize multi-task representation learning and joint optimization [5], [58]. While newly developed deep learning segmentation algorithms assumed that robust features can be extracted to simultaneously reduce speckle noise through holistic model design, loss functions, and training strategies [4], [6], [7], [16]- [18].…”