Quality-Aware Memory Network for Interactive Volumetric Image Segmentation

Abstract

Despite recent progress of automatic medical image segmentation techniques, fully automatic results usually fail to meet the clinical use and typically require further refinement. In this work, we propose a quality-aware memory network for interactive segmentation of 3D medical images. Provided by user guidance on an arbitrary slice, an interaction network is firstly employed to obtain an initial 2D segmentation. The quality-aware memory network subsequently propagates the initial segmentation estimation bidirectionally over the entire volume. Subsequent refinement based on additional user guidance on other slices can be incorporated in the same manner. To further facilitate interactive segmentation, a quality assessment module is introduced to suggest the next slice to segment based on the current segmentation quality of each slice. The proposed network has two appealing characteristics: 1) The memory-augmented network offers the ability to quickly encode past segmentation information, which will be retrieved for the segmentation of other slices; 2) The quality assessment module enables the model to directly estimate the qualities of segmentation predictions, which allows an active learning paradigm where users preferentially label the lowestquality slice for multi-round refinement. The proposed network leads to a robust interactive segmentation engine, which can generalize well to various types of user annotations (e.g., scribbles, boxes). Experimental results on various medical datasets demonstrate the superiority of our approach in comparison with existing techniques.

Segmentation of Liver Tumors by Monai and PyTorch in CT Images with Deep Learning Techniques

“…The liver and its surrounding organs have low contrast intensities, challenging liver segmentation from CT images. Recent techniques for image segmentation in medicine have used deep neural networks [16,17] and proposed a high-quality memory network for interactive 3D medical image segmentation. It uses an extended memory network to swiftly encode and retrieve segments from the past to segment new slices.…”

“…With an accuracy of 99.9, Ref. [17] classified liver tumors using a semantic pixel classification network called SegNet. Automatic segmentation may be challenging due to the liver's abdominal location and the tumor's overlap with the liver.…”

Less-than-One Shot 3D Segmentation Hijacking a Pre-trained Space-Time Memory Network

“…By combining the interactive network for the first frame and the memory network to propagate the mask, recent works produce a segmentation mask for the whole video with minimal input from the user [3]. Similar methods have been applied in volumic segmentation [30,31,16]. However, for complex porous networks imaged with ET, standard interactive methods struggle to segment correctly.…”

Segmentation of Liver Tumor in CT Scan Using ResU-Net

“…Automatic and robust liver segmentation from CT volumes is challenging due to the low-intensity contrast between the liver and neighbouring organs. Deep neural networks are used extensively in present healthcare image segmentation frameworks [10,11].Tianfei Zhou et al [12] introduced quality-aware memory network for interactive segmentation of 3D medical images. The memory-augmented network can quickly encode and retrieve segments from the past for segmentation of new slices.…”