Weakly Supervised Semantic Segmentation of Remote Sensing Images for Tree Species Classification Based on Explanation Methods

Ahlswede, Steve; Madam, Nimisha Thekke; Schulz, Christian; Kleinschmit, Birgit; Demir, Begüm

doi:10.1109/igarss46834.2022.9884676

Cited by 6 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result demonstrates the potential to map predominant tree species at the 0.2 m resolution [c.f. Ahlswede et al, 2022]. However, our class-wise model performances using mono-temporal (i.e., mono-seasonal summer scenes from different years) data could not reach the accuracies of previous studies exploiting S2 multi-temporal (i.e., multi-seasonal) data [e.g., Immitzer et al, 2019;Grabska et al, 2019].…”

Section: Discussionmentioning

confidence: 59%

“…Code and data availability. The TreeSatAI Benchmark Archive was made available through Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al, 2022] under the Creative Commons Attribution 4.0 International. Full code examples are published on the GitHub repositories of the Remote Sensing Image Analysis (RSiM) Group (https://git.tu-berlin.de/rsim/treesat_benchmark) and the Deutsches Forschungszen-…”

Section: Discussionmentioning

confidence: 99%

“…Given the potential improvement in performance from high-resolution aerial imagery and additional spectral information from Sentinel sensors, we introduce the TreeSatAI Benchmark Archive [Schulz et al, 2022]. Based on reference data from forest administration data in Germany, the dataset aims to gather multi-sensor and multi-label information for the classification of 20 tree species at Central Europe.…”

Section: Study Aimmentioning

confidence: 99%

“…The original dataset used in this paper is shared via Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al, 2022].…”

mentioning

confidence: 99%

See 3 more Smart Citations

TreeSatAI Benchmark Archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing

Ahlswede¹,

Schulz²,

Gava

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. Airborne and spaceborne platforms are the primary data sources for large-scale forest mapping, but visual interpretation for individual species determination is labour-intensive. Hence, various studies focusing on forests have investigated the benefits of multiple sensors for automated tree species classification. However, transferable deep learning approaches for large-scale applications are still lacking. This gap motivated us to create a novel dataset for tree species classification in Central Europe based on multi-sensor data from aerial, Sentinel-1 and Sentinel-2 imagery. In this paper, we introduce the TreeSatAI Benchmark Archive, which contains labels of 20 European tree species (i.e., 15 tree genera) derived from forest administration data of the federal state of Lower Saxony, Germany. We propose models and guidelines for the application of the latest machine learning techniques for the task of tree species classification with multi-label data. Finally, we provide various benchmark experiments showcasing the information which can be derived from the different sensors including artificial neural networks and tree-based machine learning methods. We found that residual neural networks (ResNet) perform sufficiently well with weighted precision scores up to 79 % only by using the RGB bands of aerial imagery. This result indicates that the spatial content present within the 0.2 m resolution data is very informative for tree species classification. With the incorporation of Sentinel-1 and Sentinel-2 imagery, performance improved marginally. However, the sole use of Sentinel-2 still allows for weighted precision scores of up to 74 % using either multi-layer perceptron (MLP) or Light Gradient Boosting Machine (LightGBM) models. Since the dataset is derived from real-world reference data, it contains high class imbalances. We found that this dataset attribute negatively affects the models' performances for many of the underrepresented classes (i.e., scarce tree species). However, the class-wise precision of the best performing late fusion model still reached values ranging from 54 % (Acer) to 88 % (Pinus). Based on our results, we conclude that deep learning techniques using aerial imagery could considerably support forestry administration in the provision of large-scale tree species maps at a very high resolution to plan for challenges driven by global environmental change. The original dataset used in this paper is shared via Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al., 2022]. For citation of the dataset, we refer to this article.

show abstract

Section: Discussionmentioning

confidence: 59%

Section: Discussionmentioning

confidence: 99%

Section: Study Aimmentioning

confidence: 99%

“…The original dataset used in this paper is shared via Zenodo (https://doi.org/10.5281/zenodo.6598390) [Schulz et al, 2022].…”

mentioning

confidence: 99%

See 2 more Smart Citations

TreeSatAI Benchmark Archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing

Ahlswede¹,

Schulz²,

Gava

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the experiments, we used three RS multi-label datasets, namely: 1) UCMerced Land Use Dataset [45] denoted as UCMerced, 2) the initial version of the TreeSatAI dataset presented in [46] denoted as TreeSatAI and 3) BigEarthNet19-Ireland dataset [47] denoted as BEN19-Ireland. Fig.…”

Section: Dataset Description and Experimental Designmentioning

confidence: 99%

On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification

Burgert

Ravanbakhsh

Demir

2022

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

The development of accurate methods for multilabel classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. To address MLC problems, the use of deep neural networks that require a high number of reliable training images annotated by multiple land-cover class labels (multi-labels) has been found popular in RS. However, collecting such annotations is time-consuming and costly. A common procedure to obtain annotations at zero labeling cost is to rely on thematic products or crowdsourced labels. As a drawback, these procedures come with the risk of label noise that can distort the learning process of the MLC algorithms. In the literature, most label noise robust methods are designed for single-label classification (SLC) problems in computer vision (CV), where each image is annotated by a single label. Unlike SLC, label noise in MLC can be associated with: 1) subtractive label noise (a land cover class label is not assigned to an image while that class is present in the image); 2) additive label noise (a land cover class label is assigned to an image, although that class is not present in the given image); and 3) mixed label noise (a combination of both). In this paper, we investigate three different noise robust CV SLC methods (Self-Adaptive Training, Early-Learning Regularization, and Joint Co-Regularized Training) and adapt them to be robust for multi-label noise scenarios in RS. During experiments, we study the effects of different types of multi-label noise and evaluate the adapted methods rigorously. To this end, we also introduce a synthetic multi-label noise injection strategy that is more adequate to simulate operational scenarios compared to the uniform label noise injection strategy, in which the labels of absent and present classes are flipped at uniform probability. Further, we study the relevance of different evaluation metrics in MLC problems under noisy multi-labels. On the basis of the theoretical and experimental analyses, some guidelines for a proper design of label noise robust MLC methods are derived.

show abstract

Sparse Multimodal Vision Transformer for Weakly Supervised Semantic Segmentation

Hanna

Mommert

Borth

2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

Weakly Supervised Semantic Segmentation of Remote Sensing Images for Tree Species Classification Based on Explanation Methods

Cited by 6 publications

References 7 publications

TreeSatAI Benchmark Archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing

TreeSatAI Benchmark Archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing

On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification

Sparse Multimodal Vision Transformer for Weakly Supervised Semantic Segmentation

Contact Info

Product

Resources

About