Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps

Audebert, Nicolas; Saux, Bertrand Le; Lefèvre, Sébastien

doi:10.1109/cvprw.2017.199

Cited by 90 publications

(66 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work of [2] investigated late fusion of Lidar and optical data for semantic segmentation using prediction fusion that required no feature engineering by combining two classifiers with a deep learning endto-end approach. This was also investigated in [36] to fuse optical and Open-StreetMap for semantic labeling. During the Data Fusion Contest (DFC) 2015, [29] proposed an early fusion scheme of Lidar and optical data based on a stack of deep features for superpixel-based classification of urban remote sensed data.…”

Section: Related Workmentioning

confidence: 99%

Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks

Audebert

Saux

Lefèvre

2018

ISPRS Journal of Photogrammetry and Remote Sensing

Self Cite

448

272

View full text Add to dashboard Cite

In this work, we investigate various methods to deal with semantic labeling of very high resolution multi-modal remote sensing data. Especially, we study how deep fully convolutional networks can be adapted to deal with multi-modal and multi-scale remote sensing data for semantic labeling. Our contributions are three-fold: a) we present an efficient multi-scale approach to leverage both a large spatial context and the high resolution data, b) we investigate early and late fusion of Lidar and multispectral data, c) we validate our methods on two public datasets with state-of-the-art results. Our results indicate that late fusion make it possible to recover errors steaming from ambiguous data, while early fusion allows for better joint-feature learning but at the cost of higher sensitivity to missing data.

show abstract

Section: Related Workmentioning

confidence: 99%

Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks

Audebert

Saux

Lefèvre

2018

ISPRS Journal of Photogrammetry and Remote Sensing

Self Cite

448

272

View full text Add to dashboard Cite

show abstract

“…When integrating HR/VHR imagery acquired at different azimuth and elevation angles, features such as building roofs show offsets similar to those caused by topography. These offsets are particularly problematic for (a) training repeated mappings of the same features, and/or (b) when using an existing vector dataset such as OpenStreetMap (OSM) as TD [133][134][135].…”

Section: Design-related Errorsmentioning

confidence: 99%

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

et al. 2020

View full text Add to dashboard Cite

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

show abstract

“…3) Third Model (v3): OpenStreetMap (OSM) 2 data is usually used for semantic labelling in RS for various purpose such as a ground truth, automatic labelling, etc. In [19], OSM is used for automatic object extraction or to build better semantic maps with the help of deep learning techniques. v3 applies the same concept.…”

Section: ) Second Model (V2)mentioning

confidence: 99%

Deep Learning Approach for Building Detection in Satellite Multispectral Imagery

Prathap

Afanasyev

2018

2018 International Conference on Intelligent Systems (IS)

View full text Add to dashboard Cite

Building detection from satellite multispectral imagery data is being a fundamental but a challenging problem mainly because it requires correct recovery of building footprints from high-resolution images. In this work, we propose a deep learning approach for building detection by applying numerous enhancements throughout the process. Initial dataset is preprocessed by 2-sigma percentile normalization. Then data preparation includes ensemble modelling where 3 models were created while incorporating OpenStreetMap data. Binary Distance Transformation (BDT) is used for improving data labeling process and the U-Net (Convolutional Networks for Biomedical Image Segmentation) is modified by adding batch normalization wrappers. Afterwards, it is explained how each component of our approach is correlated with the final detection accuracy. Finally, we compare our results with winning solutions of SpaceNet 2 competition for real satellite multispectral images of Vegas, Paris, Shanghai and Khartoum, demonstrating the importance of our solution for achieving higher building detection accuracy.

show abstract

Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps

Cited by 90 publications

References 37 publications

Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks

Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Deep Learning Approach for Building Detection in Satellite Multispectral Imagery

Contact Info

Product

Resources

About