Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention

Yan, Haotian; Zhang, Chuang; Wu, Ming C.

doi:10.48550/arxiv.2201.01615

Cited by 11 publications

(11 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is noteworthy that the RA-Net can be simply implemented by reusing the network's output with class probabilities without complex modules, while remaining faithful to the concept of attention that focuses on important RoI-associated context of the image. Moreover, it possesses advantages in terms of efficiency and performance, compared to transformer networks that prioritize efficiency like Lawin [29], SegNeXt [30], EfficientViT [31].…”

Section: Related Workmentioning

confidence: 99%

RoI-Attention Network for Small Disease Segmentation in Crop Images

Moon,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

We aim to contribute to deep learning based smart agriculture through semantic segmentation on crop images from real field environment. The key objective is the precise detection of diseases to facilitate the automation of agricultural management. The most significant issue is that the disease regions, serving as Regions of Interest (RoI), are small, making accurate prediction challenging. To address this issue, we propose a new framework of RoI-Attention Network (RA-Net) which additionally utilizes an RoI-attentive image that includes only regions predicted as disease and their surroundings from the input image. Using the RoI-attentive image, RA-Net enhances the representation power for disease regions by guiding the network to re-focus on RoI-associated context based on the initial prediction from the input. Using the proposed RoI-Attention stage, the coarse predictions of disease regions in crop images can be enhanced by incorporating additional sequential RoI-Attention and fusion stages. We have experimentally demonstrated the effectiveness of the proposed RA-Net in predicting small disease regions.

show abstract

Section: Related Workmentioning

confidence: 99%

RoI-Attention Network for Small Disease Segmentation in Crop Images

Moon,

Kim

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Because it requires a finer granularity analysis of the image, semantic segmentation is also a more time consuming and challenging task in image analysis. To achieve this, most of the neural networkbased models utilize an encoder/decoder-like architecture, such as U-Net [100], FCN [101], SegNet [102], DeepLab [103][104][105], AdaptSegNet [106], Fast-SCNN [107], HANet [108], Panoptic-deeplab [109], SegFormer [110], or Lawin+ [111]. The encoder conducts feature extraction through CNNs and derives an abstract representation (also called a feature map) of the original image.…”

Section: Semantic Segmentationmentioning

confidence: 99%

GeoAI for Large-Scale Image Analysis and Machine Vision: Recent Progress of Artificial Intelligence in Geography

Hsu

2022

IJGI

View full text Add to dashboard Cite

GeoAI, or geospatial artificial intelligence, has become a trending topic and the frontier for spatial analytics in Geography. Although much progress has been made in exploring the integration of AI and Geography, there is yet no clear definition of GeoAI, its scope of research, or a broad discussion of how it enables new ways of problem solving across social and environmental sciences. This paper provides a comprehensive overview of GeoAI research used in large-scale image analysis, and its methodological foundation, most recent progress in geospatial applications, and comparative advantages over traditional methods. We organize this review of GeoAI research according to different kinds of image or structured data, including satellite and drone images, street views, and geo-scientific data, as well as their applications in a variety of image analysis and machine vision tasks. While different applications tend to use diverse types of data and models, we summarized six major strengths of GeoAI research, including (1) enablement of large-scale analytics; (2) automation; (3) high accuracy; (4) sensitivity in detecting subtle changes; (5) tolerance of noise in data; and (6) rapid technological advancement. As GeoAI remains a rapidly evolving field, we also describe current knowledge gaps and discuss future research directions.

show abstract

“…Leveraging the advance of DETR [62], MaX-DeepLab [63] and MaskFormer [64] view image segmentation from the perspective of mask classification. Following this trend, various architectures of dense prediction transformers [65], [66], [67], [68] and semantic segmentation transformers [69], [70], [71], [72] emerge in the field. While these approaches have achieved high segmentation performance, most of them focus on using RGB images and suffer when RGB images cannot provide sufficient information in real-world scenes, e.g., under low-illumination conditions or in high-dynamic areas.…”

Section: Transformer-driven Semantic Segmentationmentioning

confidence: 99%

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers

Liu¹,

Zhang²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

The performance of semantic segmentation of RGB images can be advanced by exploiting informative features from supplementary modalities. In this work, we propose CMX, a vision-transformer-based cross-modal fusion framework for RGB-X semantic segmentation. To generalize to different sensing modalities encompassing various uncertainties, we consider that comprehensive crossmodal interactions should be provided. CMX is built with two streams to extract features from RGB images and the complementary modality (X-modality). In each feature extraction stage, we design a Cross-Modal Feature Rectification Module (CM-FRM) to calibrate the feature of the current modality by combining the feature from the other modality, in spatial-and channel-wise dimensions. With rectified feature pairs, we deploy a Feature Fusion Module (FFM) to mix them for the final semantic prediction. FFM is constructed with a cross-attention mechanism, which enables exchange of long-range contexts, enhancing both modalities' features at a global level. Extensive experiments show that CMX generalizes to diverse multi-modal combinations, achieving state-of-the-art performances on four RGB-Depth benchmarks, as well as RGB-Thermal and RGB-Polarization datasets. Besides, to investigate the generalizability to dense-sparse data fusion, we establish a RGB-Event semantic segmentation benchmark based on the EventScape dataset, on which CMX sets the new state-of-the-art. Code is available at https://github.com/huaaaliu/RGBX Semantic Segmentation

show abstract

Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention

Cited by 11 publications

References 41 publications

RoI-Attention Network for Small Disease Segmentation in Crop Images

RoI-Attention Network for Small Disease Segmentation in Crop Images

GeoAI for Large-Scale Image Analysis and Machine Vision: Recent Progress of Artificial Intelligence in Geography

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers

Contact Info

Product

Resources

About