Rethinking U-Net from an Attention Perspective with Transformers for Osteosarcoma MRI Image Segmentation

Abstract

Osteosarcoma is one of the most common primary malignancies of bone in the pediatric and adolescent populations. The morphology and size of osteosarcoma MRI images often show great variability and randomness with different patients. In developing countries, with large populations and lack of medical resources, it is difficult to effectively address the difficulties of early diagnosis of osteosarcoma with limited physician manpower alone. In addition, with the proposal of precision medicine, existing MRI image segmentation models for osteosarcoma face the challenges of insufficient segmentation accuracy and high resource consumption. Inspired by transformer’s self-attention mechanism, this paper proposes a lightweight osteosarcoma image segmentation architecture, UATransNet, by adding a multilevel guided self-aware attention module (MGAM) to the encoder-decoder architecture of U-Net. We successively perform dataset classification optimization and remove MRI image irrelevant background. Then, UATransNet is designed with transformer self-attention component (TSAC) and global context aggregation component (GCAC) at the bottom of the encoder-decoder architecture to perform integration of local features and global dependencies and aggregation of contexts to learned features. In addition, we apply dense residual learning to the convolution module and combined with multiscale jump connections, to improve the feature extraction capability. In this paper, we experimentally evaluate more than 80,000 osteosarcoma MRI images and show that our UATransNet yields more accurate segmentation performance. The IOU and DSC values of osteosarcoma are 0.922 ± 0.03 and 0.921 ± 0.04, respectively, and provide intuitive and accurate efficient decision information support for physicians.

A medical assistant segmentation method for MRI images of osteosarcoma based on DecoupleSegNet

“…The radiotherapy, chemotherapy, and surgery systems in developing countries are not complete, and the regional distribution of public medical resources is uneven. [14][15][16] In China, the large cities with only 10% of the population concentrate 80% of the medical resources. [17][18][19] It is difficult for most patients to confirm whether they are sick.…”

Auxiliary Segmentation Method of Osteosarcoma in MRI Images Based on Denoising and Local Enhancement

“…In recent years, artificial intelligence-assisted segmentation methodologies of osteosarcoma images have also been developed, including fuzzy connectivity [ 17 ], region growing, unsupervised clustering methods [ 24 ], supervised machine learning methods, etc. [ 25 , 26 , 27 ]. Using computer-aided diagnosis technology and artificial intelligence systems can ease the problems, including the shortage of medical resources, serious imbalance of doctor-patient ratio, and lack of professional doctors in developing countries.…”

A Residual Fusion Network for Osteosarcoma MRI Image Segmentation in Developing Countries

“…Cross-sectional imaging techniques such as CT scans or MRI can represent the extent of osteosarcoma invasion of bone and soft tissue vividly [ 4 ]. However, CT is less often used to scan and detect the primary tumors, since MRI can better visualize statuses like soft tissue extension, localized intramedullary metastases, and intramedullary beating metastases [ 5 ]. Therefore, doctors often make use of MRI images to provide a thorough evaluation and diagnosis of patients.…”