Deep Learning for Automatic Segmentation of Hybrid Optoacoustic Ultrasound (OPUS) Images

Abstract

The highly complementary information provided by multispectral optoacoustics and pulseecho ultrasound have recently prompted development of hybrid imaging instruments bringing together the unique contrast advantages of both modalities. In the hybrid optoacoustic ultrasound (OPUS) combination, images retrieved by one modality may further be used to improve the reconstruction accuracy of the other. In this regard, image segmentation plays a major role as it can aid improving the image quality and quantification abilities by facilitating modeling of light and sound propagation through the imaged tissues and surrounding coupling medium. Here, we propose an automated approach for surface segmentation in whole-body mouse OPUS imaging using a deep convolutional neural network (CNN). The method has shown robust performance, attaining accurate segmentation of the animal boundary in both optoacoustic and pulse-echo ultrasound images, as evinced by quantitative performance evaluation using Dice coefficient metrics.

Random Fourier Features-Based Deep Learning Improvement with Class Activation Interpretability for Nerve Structure Segmentation

“…(ii) A CAMbased extension for semantic segmentation (see Section 2.3) from RFF-based mappings is proposed to highlight the most relevant features (image regions) that favor discriminating between nerve and background. Figure 3 For concrete testing, we apply our RFF-based proposal within the FCN [12], U-net [14,41], and ResUnet [16] approaches. Our main aim is to improve the representation learning benefits of deep models using robust kernel mappings.…”

“…Concerning this, an RFF-based approach is employed to approximate a Gaussian kernel implicit mapping within three wellknown architectures for image-based semantic segmentation. In particular, the FCN [12], Unet [14,41], and ResUnet [16] are studied. Our RFF-based improvement aims to provide a better generalization capability for ultrasound image-based nerve segmentation using straightforward and complex architectures.…”

Full-view in vivo skin and blood vessels profile segmentation in photoacoustic imaging based on deep learning

“…Another important review article by Grohl et al [20] presented the current advancement regarding DL in PA imaging. DL has been determined to be applicable in a variety of aspects for PAM such as image reconstruction [21] , [22] , [23] , [24] , [25] , image classification [26] , quantitative imaging [27] , image detection, [28] and, especially, image segmentation [26] , [29] , [30] , [31] , [32] , [33] . However, those studies almost have been reported about segmentation of the C-scan image (MAP image domain) [26] , [29] , [31] , [32] , [33] , not widely applied to 3D PA images for separating skin and blood vessels areas.…”

“…DL has been determined to be applicable in a variety of aspects for PAM such as image reconstruction [21] , [22] , [23] , [24] , [25] , image classification [26] , quantitative imaging [27] , image detection, [28] and, especially, image segmentation [26] , [29] , [30] , [31] , [32] , [33] . However, those studies almost have been reported about segmentation of the C-scan image (MAP image domain) [26] , [29] , [31] , [32] , [33] , not widely applied to 3D PA images for separating skin and blood vessels areas. Unlike other works, Chlis et al [30] used a DL method for processing the cross-sectional B-scan image to avoid the rigorous and time-consuming manual segmentation.…”

Binary photoacoustic tomography for improved vasculature imaging

“… 22 Further, deep learning-based approach was proposed for segmenting animal boundary using multi-modal photoacoustic and ultrasound data. 23 Further, deep learning-based method has also been developed for animal brain imaging. 24 …”