UMSSNet: A Unified Multi-Scale SegmentationNetwork for Heterogeneous Medical Images

Abstract

Medical image analysis plays a pivotal role in diagnosis and treatment. However, the diverse characteristics of various imaging modalities often demand distinct processing approaches. In this study, we introduce UMSSNet, a versatile method for medical image segmentation that leverages data from heterogeneous medical images across different scales. By utilizing Gaussian pyramid-based image processing techniques, we transform the heterogeneous medical images into a uniform multi-scale image structure. Subsequently, UMSSNet integrates multiscale image features, encompassing contextual information, and adopts a dynamic and hierarchical approach to process images at various scales, emulating the decision-making process of human pathologists and facilitating precise image segmentation. We tested UMSSNet on publicly available datasets consisting of various forms of medical images, including WSI, Biopsy slides, CT, MRI, X-ray, Colonoscopy, Fundus, and CMR, as well as private datasets of Immunohistochemical staining, Immunofluorescence staining, and Masson staining sample images. UMSSNet demonstrated performance comparable to state-of-the-art medical image segmentation methods Furthermore, the generalizability of UMSSNet in segmenting heterogeneous medical images holds promise for future research in the analysis of multimodal medical data.