CSE-HRNet: A Context and Semantic Enhanced High-Resolution Network for Semantic Segmentation of Aerial Imagery

Wang, Fang; Piao, Shihao; Xie, Jindong

doi:10.1109/access.2020.3028445

Cited by 16 publications

(16 citation statements)

References 51 publications

(103 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since DL semantic and instance segmentation algorithm development and refinement are still actively being studied, there will be a continued need to investigate these new and refined methods for a wide variety of mapping tasks. For example, high-resolution networks (HRNets) [94][95][96][97] have recently been shown to be of value for dealing with issues of intra-class heterogeneity and inter-class homogeneity. Further, gated shape CNNs have been shown to be useful for differentiating features based on unique shape characteristics [98,99].…”

Section: Discussionmentioning

confidence: 99%

Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps

et al. 2020

View full text Add to dashboard Cite

Historic topographic maps, which are georeferenced and made publicly available by the United States Geological Survey (USGS) and the National Map’s Historical Topographic Map Collection (HTMC), are a valuable source of historic land cover and land use (LCLU) information that could be used to expand the historic record when combined with data from moderate spatial resolution Earth observation missions. This is especially true for landscape disturbances that have a long and complex historic record, such as surface coal mining in the Appalachian region of the eastern United States. In this study, we investigate this specific mapping problem using modified UNet semantic segmentation deep learning (DL), which is based on convolutional neural networks (CNNs), and a large example dataset of historic surface mine disturbance extents from the USGS Geology, Geophysics, and Geochemistry Science Center (GGGSC). The primary objectives of this study are to (1) evaluate model generalization to new geographic extents and topographic maps and (2) to assess the impact of training sample size, or the number of manually interpreted topographic maps, on model performance. Using data from the state of Kentucky, our findings suggest that DL semantic segmentation can detect surface mine disturbance features from topographic maps with a high level of accuracy (Dice coefficient = 0.902) and relatively balanced omission and commission error rates (Precision = 0.891, Recall = 0.917). When the model is applied to new topographic maps in Ohio and Virginia to assess generalization, model performance decreases; however, performance is still strong (Ohio Dice coefficient = 0.837 and Virginia Dice coefficient = 0.763). Further, when reducing the number of topographic maps used to derive training image chips from 84 to 15, model performance was only slightly reduced, suggesting that models that generalize well to new data and geographic extents may not require a large training set. We suggest the incorporation of DL semantic segmentation methods into applied workflows to decrease manual digitizing labor requirements and call for additional research associated with applying semantic segmentation methods to alternative cartographic representations to supplement research focused on multispectral image analysis and classification.

show abstract

Section: Discussionmentioning

confidence: 99%

Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Compared with natural images, semantic segmentation for aerial images is more challenging, as the aerial images are captured from a bird's view with less common structural information and few spectral channels. This aggravates the inter-class homogeneity and intra-class heterogeneity [6], [7] problems.…”

Section: Semantic Segmentation Of Aerial Imagerymentioning

confidence: 99%

“…In semantic segmentation of high resolution aerial images, inter-class homogeneity and intra-class heterogeneity are two issues [6], [7]. On one hand, the aerial images are acquired from a bird's view with less common structural information, which could make the inter-class homogeneity of land covers worse.…”

Section: Introductionmentioning

confidence: 99%

CCENet: Cascade Class-Aware Enhanced Network for High-Resolution Aerial Imagery Semantic Segmentation

Wang

Luo

Feng

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Semantic segmentation of high-resolution aerial images is a challenging task on account of inter-class homogeneity and intra-class heterogeneity of land cover. Recent works have sought to mitigate this issue by exploiting pixel-wise global contextual information using self-attention mechanism. However, the existing attention-based methods usually generate inaccurate object boundary segmentation results, as the selfattention model is embedded in high-level features with low resolution due to prohibitively computational complexity. Moreover, existing attention-based models ignore class-wise contextual information from intermediate results, which leads to undesirable feature separability. To obtain discriminative feature as well as generate accurate segmentation boundaries, we present a novel segmentation framework, named Cascade Class-aware Enhanced Network (CCENet) for high resolution aerial imagery. The proposed CCENet predicts segmentation results on multiple stages, and the result of the previous stage is used to refine object boundary details for the latter stage. To exploit the class-aware prior information in previous stage, we propose a lightweight Class-aware Enhanced Module (CaEM) to grab the class-aware contextual dependencies. Specifically, CaEM first extract a set of class representation of the land covers by Global Class Pooling (GCP) block, then reconstruct enhanced feature using Class Relation Measurement (CRM), which alleviates the inter-class homogeneity and intra-class heterogeneity of ground objects in feature space. Quantitative and qualitative experimental results on three publicly available datasets demonstrate the superiority of our CCENet over other state-of-the-art methods in the items of high labelling accuracy and computation efficiency.Q2 Q3

show abstract

“…Some improvements are also made to give full paly to HRNet. For example, Huo et al [58] adopt the attention to suppress the unimportant information and Wang et al [59] additionally consider the multi-level semantic information. In the task of image restoration and enhancement, Zamir et al follow such design principle and further propose the MIRNet [2].…”

Section: Multi-resolution Strategymentioning

confidence: 99%

Image quality enhancement using hybrid attention networks

Wang

Chen

Yan

2021

IET Image Processing

View full text Add to dashboard Cite

Image quality enhancement aims to recover rich details from degraded images, which is applied into many fields, such as medical imaging, filming production and autonomous driving. Deep convolutional neural networks (CNNs) have enabled rapid development of image quality enhancement. However, most existing CNN-based methods lack versatility when targeting different subtasks in terms of the design of networks. Besides, they often fail to balance precise spatial representations and necessary contextual information. To deal with these problems, this paper proposes a novel unified framework for low-light image enhancement, image denoising and image super-resolution. The core of this architecture is a residual hybrid attention block (RHAB), which consists of several dynamic down-sampling modules (DDM) and hybrid attention up-sampling modules (HAUM). Specifically, multi-scale feature maps are fully interacted with each other with the help of nested subnetworks so that both high-resolution spatial details and high-level contextual information can be combined to improve the representation ability of the network. Further, a hybrid attention network (HAN) is proposed and evaluations on three separate subtasks demonstrate its good performance. Extensive experiments on the authors' synthetic dataset, a more complex version, show that the authors' method achieve better quantitative and visual results compared to other state-of-the-art methods.

show abstract

CSE-HRNet: A Context and Semantic Enhanced High-Resolution Network for Semantic Segmentation of Aerial Imagery

Cited by 16 publications

References 51 publications

Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps

Semantic Segmentation Deep Learning for Extracting Surface Mine Extents from Historic Topographic Maps

CCENet: Cascade Class-Aware Enhanced Network for High-Resolution Aerial Imagery Semantic Segmentation

Image quality enhancement using hybrid attention networks

Contact Info

Product

Resources

About