Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu, Xiao Xiang; Tuia, Devis; Mou, Lichao; Xia, Gui-Song; Zhang, Liangpei; Xu, Feng; Fraundorfer, Friedrich

doi:10.1109/mgrs.2017.2762307

Cited by 2,238 publications

(1,223 citation statements)

References 156 publications

Supporting

Mentioning

1,204

Contrasting

Unclassified

Order By: Relevance

“…These methods have proven particularly beneficial for modeling physical relationships that are complicated, cannot be generalized or are not well-understood [18]. Thus, deep learning is potentially well suited to approximate models of phenological changes, which depend on complex internal biochemical processes of which only the change of surface reflectivity can be observed by EO sensors.…”

Section: Related Workmentioning

confidence: 99%

Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

Rußwurm

Körner

2018

IJGI

230

184

View full text Add to dashboard Cite

Earth observation (EO) sensors deliver data at daily or weekly intervals. Most land use and land cover classification (LULC) approaches, however, are designed for cloud-free and mono-temporal observations. The increasing temporal capabilities of today's sensors enable the use of temporal, along with spectral and spatial features.Domains such as speech recognition or neural machine translation, work with inherently temporal data and, today, achieve impressive results by using sequential encoder-decoder structures. Inspired by these sequence-to-sequence models, we adapt an encoder structure with convolutional recurrent layers in order to approximate a phenological model for vegetation classes based on a temporal sequence of Sentinel 2 (S2) images. In our experiments, we visualize internal activations over a sequence of cloudy and non-cloudy images and find several recurrent cells that reduce the input activity for cloudy observations. Hence, we assume that our network has learned cloud-filtering schemes solely from input data, which could alleviate the need for tedious cloud-filtering as a preprocessing step for many EO approaches. Moreover, using unfiltered temporal series of top-of-atmosphere (TOA) reflectance data, our experiments achieved state-of-the-art classification accuracies on a large number of crop classes with minimal preprocessing, compared to other classification approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

Rußwurm

Körner

2018

IJGI

230

184

View full text Add to dashboard Cite

show abstract

“…Convolutional Neural Networks (CNNs)-based models have demonstrated impressive accuracies in object recognition and image classification in the field of computer vision ( [13][14][15][16] and are starting to be used in the field of remote sensing ( [17]). This success is due to the availability of larger training datasets, better algorithms, improved network architectures, faster GPUs and also improvement techniques such as data-augmentation and transfer-learning, which allow reutilization of the knowledge acquired from a set of images into other new images.…”

Section: Introductionmentioning

confidence: 99%

Deep-learning Versus OBIA for Scattered Shrub Detection with Google Earth Imagery: Ziziphus lotus as Case Study

et al. 2017

View full text Add to dashboard Cite

There is a growing demand for accurate high-resolution land cover maps in many fields, e.g., in land-use planning and biodiversity conservation. Developing such maps has been traditionally performed using Object-Based Image Analysis (OBIA) methods, which usually reach good accuracies, but require a high human supervision and the best configuration for one image often cannot be extrapolated to a different image. Recently, deep learning Convolutional Neural Networks (CNNs) have shown outstanding results in object recognition in computer vision and are offering promising results in land cover mapping. This paper analyzes the potential of CNN-based methods for detection of plant species of conservation concern using free high-resolution Google Earth TM images and provides an objective comparison with the state-of-the-art OBIA-methods. We consider as case study the detection of Ziziphus lotus shrubs, which are protected as a priority habitat under the European Union Habitats Directive. Compared to the best performing OBIA-method, the best CNN-detector achieved up to 12% better precision, up to 30% better recall and up to 20% better balance between precision and recall. Besides, the knowledge that CNNs acquired in the first image can be re-utilized in other regions, which makes the detection process very fast. A natural conclusion of this work is that including CNN-models as classifiers, e.g., ResNet-classifier, could further improve OBIA methods. The provided methodology can be systematically reproduced for other species detection using our codes available through (https://github.com/EGuirado/CNN-remotesensing).

show abstract

“…Deep learning is gaining increasing interest in remote sensing as an approach for development and enhancement of mapping applications [36]. Remote sensing presents some new challenges for deep learning because it aims at retrieving geo-physical or bio-chemical quantities rather than detecting or recognizing objects.…”

Section: Discussionmentioning

confidence: 99%

Assessment of Convolution Neural Networks for Surficial Geology Mapping in the South Rae Geological Region, Northwest Territories, Canada

2018

View full text Add to dashboard Cite

Abstract:Mapping of surficial geology is an important requirement for broadening the geoscience database of northern Canada. Surficial geology maps are an integral data source for mineral and energy exploration. Moreover, they provide information such as the location of gravels and sands, which are important for infrastructure development. Currently, surficial geology maps are produced through expert interpretation of aerial photography and field data. However, interpretation is known to be subjective, labour-intensive and difficult to repeat. The expert knowledge required for interpretation can be challenging to maintain and transfer. In this research, we seek to assess the potential of deep neural networks to aid surficial geology mapping by providing an objective surficial materials initial layer that experts can modify to speed map development and improve consistency between mapped areas. Such an approach may also harness expert knowledge in a way that is transferable to unmapped areas. For this purpose, we assess the ability of convolution neural networks (CNN) to predict surficial geology classes under two sampling scenarios. In the first scenario, a CNN uses samples collected over the area to be mapped. In the second, a CNN trained over one area is then applied to locations where the available samples were not used in training the network. The latter case is important, as a collection of in situ training data can be costly. The evaluation of the CNN was carried out using aerial photos, Landsat reflectance, and high-resolution digital elevation data over five areas within the South Rae geological region of Northwest Territories, Canada. The results are encouraging, with the CNN generating average accuracy of 76% when locally trained. For independent test areas (i.e., trained over one area and applied over other), accuracy dropped to 59-70% depending on the classes selected for mapping. In the South Rae region, significant confusion was found between till veneer and till blanket as well as glaciofluvial subclasses (esker, terraced, and hummocky ice-contact). Merging these classes respectively increased accuracy for independent test area to 68% on average. Relative to the more widely used Random Forest machine learning algorithm, this represents an improvement in accuracy of 4%. Furthermore, the CNN produced better results for less frequent classes with distinct spatial structure.

show abstract

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Cited by 2,238 publications

References 156 publications

Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

Deep-learning Versus OBIA for Scattered Shrub Detection with Google Earth Imagery: Ziziphus lotus as Case Study

Assessment of Convolution Neural Networks for Surficial Geology Mapping in the South Rae Geological Region, Northwest Territories, Canada

Contact Info

Product

Resources

About