Parameter-Free Latent Space Transformer for Zero-Shot Bidirectional Cross-modality Liver Segmentation

Li, Yang; Zou, Beiji; Dai, Yulan; Zhu, Chengzhang; Yang, Fan; Li, Xin; Bai, Harrison X.; Jiao, Zhicheng

doi:10.1007/978-3-031-16440-8_59

Cited by 1 publication

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chen introduced latent variables to provide a hierarchical generation approach, where the high-level semantics are determined by the latent variables and then decoded to generate sentence-level syntax and lexical details, thus achieving a generative conversational network (Chen et al 2022). Li proposed a parameter-free latent space transformer (LST) based on an intermediate generative model and learning domain-invariant features, and investigated a shared Latent Space that maximizes the similarity between CT and MR liver images while minimizing domain shift, achieving satisfactory zero-shot bidirectional cross-modal liver segmentation performance (Li et al 2022). These studies have fully demonstrated the effectiveness of Latent Space in feature interaction, especially for feature learning across different modalities.…”

Section: Introductionmentioning

confidence: 99%

LSAM: L2-norm self-attention and latent space feature interaction for automatic 3D multi-modal head and neck tumor segmentation

Li,

Tan,

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective. Head and Neck (H&N) cancers are prevalent globally, and early and accurate detection is absolutely crucial for timely and effective treatment. However, the segmentation of H&N tumors is challenging due to the similar density of the tumors and surrounding tissues in CT images. While PET images provide information about the metabolic activity of the tissue and can distinguish between lesion regions and normal tissue. But they are limited by their low spatial resolution. To fully leverage the complementary information from PET and CT images, we propose a novel and innovative multi-modal tumor segmentation method specifically designed for H\&N tumor segmentation. Approach. The proposed novel and innovative multi-modal tumor segmentation network (LSAM) consists of two key learning modules, namely L2-Norm Self-Attention and Latent Space feature interaction, which exploit the high sensitivity of PET images and the anatomical information of CT images. These two advanced modules contribute to a powerful 3D segmentation network based on a U-shaped structure. The well-designed segmentation method can integrate complementary features from different modalities at multiple scales, thereby improving the feature interaction between modalities.Main results. We evaluated the proposed method on the public HECKTOR PET-CT dataset, and the experimental results demonstrate that the proposed method convincingly outperforms existing H&N tumor segmentation methods in terms of key evaluation metrics, including DSC (0.8457), Jaccard (0.7756), RVD (0.0938), and HD95 (11.75).  Significance. The innovative Self-Attention mechanism based on L2-Norm offers scalability and is effective in reducing the impact of outliers on the performance of the model. And the novel method for multi-scale feature interaction based on Latent Space utilizes the learning process in the encoder phase to achieve the best complementary effects among different modalities. Our code will be publicly available on https://zhengshenhai.github.io/.

show abstract

Section: Introductionmentioning

confidence: 99%