Multiresolution Aggregation Transformer UNet Based on Multiscale Input and Coordinate Attention for Medical Image Segmentation

Chen, Shao-Long; Qiu, Changzhen; Yang, Weiping; Zhang, Zhiyong

doi:10.3390/s22103820

Cited by 17 publications

(10 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the precise segmentation of a right ventricle (RV) structure demands both short‐axis (SA) images and long‐axis (LA) images, posing a challenge for the current segmentation methods. Fusing the U‐Net with Transformer, Chen et al 54 proposed an MRA‐TUNet to achieve the segmentation of atrial and ventricle. The performance of MRA‐TUNet was confirmed by the experimental results on ACDC and 2018 atrial segmentation challenge.…”

Section: Medical Image Segmentationmentioning

confidence: 99%

Recent advances of Transformers in medical image analysis: A comprehensive review

Xia

Wang

2023

MedComm – Future Medicine

View full text Add to dashboard Cite

Recent works have shown that Transformer's excellent performances on natural language processing tasks can be maintained on natural image analysis tasks. However, the complicated clinical settings in medical image analysis and varied disease properties bring new challenges for the use of Transformer. The computer vision and medical engineering communities have devoted significant effort to medical image analysis research based on Transformer with especial focus on scenario-specific architectural variations.In this paper, we comprehensively review this rapidly developing area by covering the latest advances of Transformer-based methods in medical image analysis of different settings. We first give introduction of basic mechanisms of Transformer including implementations of selfattention and typical architectures. The important research problems in various medical image data modalities, clinical visual tasks, organs and diseases are then reviewed systemically. We carefully collect 276 very recent works and 76 public medical image analysis datasets in an organized structure. Finally, discussions on open problems and future research directions are also provided. We expect this review to be an up-to-date roadmap and serve as a reference source in pursuit of boosting the development of medical image analysis field.

show abstract

Section: Medical Image Segmentationmentioning

confidence: 99%

Recent advances of Transformers in medical image analysis: A comprehensive review

Xia

Wang

2023

MedComm – Future Medicine

View full text Add to dashboard Cite

show abstract

“…The proposed transformer encoder block with both attention mechanism and convolution layer is implanted in the U-Net. Chen et al [112] designed a U-shape model named MRA-TUNet to segment MRI images [113,114].…”

Section: Segmentationmentioning

confidence: 99%

“…The proposed model is composed of a down-sample part, an up-sample part, and a connection part. The UTNet [110] 2021 MRI left ventricle, right ventricle, left ventricular myocardium [111] MRA-TUNet [112] 2022 MRI left ventricle, right ventricle, left ventricular myocardium, left atrium cardiac disease [113], atrial fibrillation [114] HybridCTrm [115] 2021 MRI brain [116], neurodevelopmental disorders [117] consistency-based co-segmentation [118] 2021 MRI right ventricle [119] TransConver [120] 2022 MRI brain brain tumor [121,122,123] UTransNet [124] 2022 MRI brain stroke [129] TransBTS [125] 2021 MRI brain brain tumor [121,122,123] METrans [126] 2022 MRI brain stroke [130], ischemic stroke lesion [131], schemic stroke lesion [132] SwinBTS [127] 2022 MRI brain brain tumor [121,123,133,134] BTSwin-Unet [128] 2022 MRI brain brain tumor [121,122] CVT-Vnet [135] 2022 CT head, neck organs at risk [136] CoTr [137] 2021 CT abdomen colorectal cancer, ventral hernia [138] transformer-UNet [139] 2021 CT lung [140] AFTer-UNet [141] 2022 CT abdomen, thorax [142], organs at risk…”

Section: Segmentationmentioning

confidence: 99%

Recent Progress in Transformer-based Medical Image Analysis

Liu¹,

Shen²

2022

Preprint

View full text Add to dashboard Cite

The transformer has dominated the natural language processing (NLP) field for a long time. Recently, the transformer-based method has been adopted into the computer vision (CV) field and shows promising results. As an important branch of the CV field, medical image analysis joins the wave of the transformer-based method rightfully. In this review, we illustrate the principle of the attention mechanism, and the detailed structures of the transformer, and depict how the transformer is adopted into medical image analysis. We organize the transformer-based medical image analysis applications in a sequence of different tasks, including classification, segmentation, synthesis, registration, localization, detection, captioning, and denoising. For the mainstream classification and segmentation tasks, we further divided the corresponding works based on different medical imaging modalities. The datasets corresponding to the related works are also organized. We include thirteen modalities and more than twenty objects in our work.

show abstract

“…It can also be used to process a series of 2D images with sequential relationships by converting the temporal parameter t into a spatial parameter, with t denoting the slice order [ 23 ]. In addition, PSPNet [ 24 ] utilizes convolutional neural networks with pyramidal pooling to capture target features at different scales [ 25 ] and is widely used in image segmentation problems in medicine, agriculture, and geography. The strengths and weaknesses of the common methods in left atrial segmentation are shown in Table 1 .…”

Section: Introductionmentioning

confidence: 99%

Cardiac Magnetic Resonance Image Segmentation Method Based on Multi-Scale Feature Fusion and Sequence Relationship Learning

Zuo

et al. 2023

Sensors

View full text Add to dashboard Cite

Accurate segmentation of the left atrial structure using magnetic resonance images provides an important basis for the diagnosis of atrial fibrillation (AF) and its treatment using robotic surgery. In this study, an image segmentation method based on sequence relationship learning and multi-scale feature fusion is proposed for 3D to 2D sequence conversion in cardiac magnetic resonance images and the varying scales of left atrial structures within different slices. Firstly, a convolutional neural network layer with an attention module was designed to extract and fuse contextual information at different scales in the image, to strengthen the target features using the correlation between features in different regions within the image, and to improve the network’s ability to distinguish the left atrial structure. Secondly, a recurrent neural network layer oriented to two-dimensional images was designed to capture the correlation of left atrial structures in adjacent slices by simulating the continuous relationship between sequential image slices. Finally, a combined loss function was constructed to reduce the effect of positive and negative sample imbalance and improve model stability. The Dice, IoU, and Hausdorff distance values reached 90.73%, 89.37%, and 4.803 mm, respectively, based on the LASC2013 (left atrial segmentation challenge in 2013) dataset; the corresponding values reached 92.05%, 89.41% and 9.056 mm, respectively, based on the ASC2018 (atrial segmentation challenge at 2018) dataset.

show abstract

Multiresolution Aggregation Transformer UNet Based on Multiscale Input and Coordinate Attention for Medical Image Segmentation

Cited by 17 publications

References 38 publications

Recent advances of Transformers in medical image analysis: A comprehensive review

Recent advances of Transformers in medical image analysis: A comprehensive review

Recent Progress in Transformer-based Medical Image Analysis

Cardiac Magnetic Resonance Image Segmentation Method Based on Multi-Scale Feature Fusion and Sequence Relationship Learning

Contact Info

Product

Resources

About