MLRSNet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding

Qi, Xiaoman; Zhu, Peisheng; Wang, Yuebin; Zhang, Liqiang; Peng, Junhuan; Wu, Mengfan; Chen, Jialong; Zhao, Xudong; Zang, Ning; Mathiopoulos, P. Takis

doi:10.1016/j.isprsjprs.2020.09.020

Cited by 85 publications

(43 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We start with the recent MLRSNet [42] as our base dataset. In its full form, it is composed of 109,161 images that are approximately uniformly distributed across 46 scene categories (e.g., airport, desert, and freeway).…”

Section: Test Environments a Scene Classification In Eo Satellite Imagerymentioning

confidence: 99%

Improving Out-of-Distribution Detection by Learning From the Deployment Environment

Inkawhich

Zhang

Davis

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recognition systems in the remote sensing domain often operate in "open-world" environments, where they must be capable of accurately classifying data from the indistribution categories while simultaneously detecting and rejecting anomalous/out-of-distribution (OOD) inputs. However, most modern designs use Deep Neural Networks (DNNs) to perform this recognition function that are trained under "closedworld" assumptions in offline-only environments. As a result, by construction these systems are ill-posed to handle anomalous inputs and have no mechanism for improving OOD detection abilities during deployment. In this work, we address these weaknesses from two aspects. First, we introduce advanced DNN training methods to co-design for accuracy and OOD detection in the offline training phase. We then propose a novel "learn-online" workflow for updating the DNNs during deployment using a small library of carefully collected samples from the operating environment. To show the efficacy of our methods we consider experimenting with two popular recognition tasks in remote sensing: scene classification in electro-optical satellite images and automatic target recognition in synthetic aperture radar imagery. In both, we find that our two primary design contributions can individually improve detection performance, while also being complementary. Additionally, we find that detection performance on difficult and highly-granular OOD samples can be drastically improved using only tens or hundreds of samples collected from the environment. Finally, through analysis we determine that the logic for adding/removing samples from the collection library is of key importance and using a proper learning rate during the model update step is critical.

show abstract

Section: Test Environments a Scene Classification In Eo Satellite Imagerymentioning

confidence: 99%

Improving Out-of-Distribution Detection by Learning From the Deployment Environment

Inkawhich

Zhang

Davis

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…It is obvious that the above mentioned datasets prefer to advance interpretation algorithms with limited semantic cate- [70], Patternet [73], Optimal-31 [76], fWoM [74], CLRS [78], MLRSNet [79], xVew [95], SEN12MS [130] and SECOND [131], SkyScapes [132], emphasizing broadly on scene-, object-, and pixel-level information. Even with enriched semantic categories, to fully interpret the content of interest in RS images still remains difficult.…”

Section: B Annotated Datasets For Rs Image Interpretationmentioning

confidence: 99%

On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID

Yang

Xia

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

110

View full text Add to dashboard Cite

The past years have witnessed great progress on remote sensing (RS) image interpretation and its wide applications. With RS images becoming more accessible than ever before, there is an increasing demand for the automatic interpretation of these images. In this context, the benchmark datasets serve as essential prerequisites for developing and testing intelligent interpretation algorithms. After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. Specifically, we first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. We then present the general guidances on creating benchmark datasets in efficient manners. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million-AID 1 , a new large-scale benchmark dataset containing a million instances for RS image scene classification. Several challenges and perspectives in RS image annotation are finally discussed to facilitate the research in benchmark dataset construction. We do hope this paper will provide the RS community an overall perspective on constructing large-scale and practical image datasets for further research, especially datadriven ones.

show abstract

“…J. Kang et al [18] proposed a new graph relation network (GRN) for multi-label RS scene classification, which can use a graph structure input to network learning to model the relationship between scenes. Recently, Qi et al [32] constructed a multi-label high-spatial-resolution remote sensing dataset named MLRSNet for semantic scene understanding for the task of image classification and image retrieval. Sumbul et al [33] proposed BigEarthNet-MM dataset which is made up of 590,326 pairs of Sentinel-1 and Sentinel-2 image patches.…”

Section: Multi-label Retrieval Of Remote Sensing Imagesmentioning

confidence: 99%

A Semantic-Preserving Deep Hashing Model for Multi-Label Remote Sensing Image Retrieval

Cheng

Huang

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Conventional remote sensing image retrieval (RSIR) systems perform single-label retrieval with a single label to represent the most dominant semantic content for an image. Improved spatial resolution dramatically boosts the remote sensing image scene complexity, as a remote sensing image always contains multiple categories of surface features. In this case, a single label cannot comprehensively describe the semantic content of a complex remote sensing image scene and therefore results in poor retrieval performance in practical applications. As a result, researchers have begun to pay attention to multi-label image retrieval. However, in the era of massive remote sensing data, how to increase retrieval efficiency and reduce feature storage while preserving semantic information remains unsolved. Considering the powerful capability of hashing learning in overcoming the curse of dimensionality caused by high-dimensional image representation in Approximate Nearest Neighbor (ANN) search problems, we propose a new semantic-preserving deep hashing model for multi-label remote sensing image retrieval. Our model consists of three main components: (1) a convolutional neural network to extract image features; (2) a hash layer to generate binary codes; (3) a new loss function to better maintain the multi-label semantic information of hash learning contained in context remote sensing image scene. As far as we know, this is the first attempt to apply deep hashing into the multi-label remote sensing image retrieval. Experimental results indicate the effectiveness and promising of the introduction of hashing methods in the multi-label remote sensing image retrieval.

show abstract

MLRSNet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding

Cited by 85 publications

References 49 publications

Improving Out-of-Distribution Detection by Learning From the Deployment Environment

Improving Out-of-Distribution Detection by Learning From the Deployment Environment

On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID

A Semantic-Preserving Deep Hashing Model for Multi-Label Remote Sensing Image Retrieval

Contact Info

Product

Resources

About