Mutually enhanced multi-view information learning for segmentation of lung tumor in CT images

Abstract

The accurate automatic segmentation of tumors from computed tomography (CT) volumes facilitates early diagnosis and treatment of patients. A significant challenge in tumor segmentation is the integration of the spatial correlations among multiple parts of the CT volume and the context relationship across multiple channels. We proposed a mutually enhanced multi-view information model (MEMI) to propagate and fuse the spatial correlations and the context relationship and then apply it to lung tumor segmentation from CT volumes. First, an attention mechanism from the region node perspective was presented to determine the impact of all the other nodes on a specific node, which aims to enhance the node attribute embedding. A gated convolution-based strategy was also designed to integrate the enhanced attributes and the original node features. Second, transformer across multiple channels was constructed to learn the context relationship between these channels and to fuse the information across the channels. Third, since the encoded node attributes from the gated convolution view and those from the channel transformer view were complementary, an interaction attention mechanism was proposed to propagate and fuse the mutual information from multiple views. Finally, the node embeddings with the spatial correlations, the channel context embeddings, and the original features of region nodes were adaptively integrated before sending through the segmentation decoder for final output. The segmentation performance was evaluated on both public lung tumor dataset and private dataset collected from a hospital. The experimental results demonstrated that MEMI was superior to other compared segmentation methods. Ablation studies showed the contributions of node correlation learning, channel context relationship learning, and mutual information interaction across multiple views to the improved segmentation performance. The experimental results of using MEMI on multiple segmentation backbones also demonstrated MEMI’s generalization ability.