Spatio-Temporal Consistency and Negative Label Transfer for 3D Freehand US Segmentation

Abstract

The manual segmentation of multiple organs in 3D ultrasound (US) sequences and volumes towards their quantitative analysis is very expensive and time-consuming. Fully supervised segmentation methods still require the collection of large volumes of annotated data while unlabeled images are abundant. In this work, we propose a semi-automatic deep learning approach modeled as a weak-label learning problem: given a few 2-D incomplete annotations for selected slices, the goal is to propagate the masks to the entire sequence. To this end, we make use of both positive and negative constraints induced by incomplete labels to penalize the segmentation loss function. Our model is composed of one encoder and two decoders to model the segmentation and an auxiliary reconstruction task. Moreover, we consider the spatiotemporal information by deploying a Convolutional Long Short Term Memory module. Our findings suggest that the reconstruction decoder and the spatio-temporal information lead to a better geometrical estimation of the mask shape. We apply the model to the task of low-limb muscle segmentation in a dataset of 44 patients and 6160 images.

Towards Clinical Application of Artificial Intelligence in Ultrasound Imaging

“…The iterative transformation network was proposed to guide the current plane towards the location of the standard scanning planes in the 3D US volume [61]. Moreover, Duque et al proposed a semi-automatic segmentation algorithm for a freehand 3D US volume, which is a continuum of 2D cross-sections, by employing an encoder-decoder architecture with 2D US images and several 2D labels [62]. We summarize the abovementioned segmentation algorithms for US imaging analysis in Table 2.…”

IFSS-Net: Interactive Few-Shot Siamese Network for Faster Muscle Segmentation and Propagation in Volumetric Ultrasound

“…However, ASC may also produce less sharp masks at the boundaries. Prior work has handled this issue with auxiliary refinement [30] or reconstruction [16] tasks. Herein, we rely on a series of 3D ASC (applied to the xyz planes) connected in a recurrently fashion to interpret the full context, while propagating contextual information in the bidirectional z-direction with the Bi-CLSTM.…”

“…The interest has recently shifted towards learning from a limited quantity of annotated data, using, for instance, few-shot learning [13] or self-supervision [14]. To learn representations from unlabelled input data, self-learning methods commonly rely on auxiliary tasks, such as image reconstruction [15], [16] or context restoration [17]. Self-supervision may also exploit pseudo-labelling [18] wherein unannotated data are relabelled and reused to fine-tune the trained model.…”