A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification

Abstract

The contextual-based convolutional neural network (CNN) with deep architecture and pixel-based multilayer perceptron (MLP) with shallow structure are well-recognized neural network algorithms, representing the state-of-the-art deep learning method and the classical non-parametric machine learning approach, respectively. The two algorithms, which have very different behaviours, were integrated in a concise and effective way using a rule-based decision fusion approach for the classification of very fine spatial resolution (VFSR) remotely sensed imagery. The decision fusion rules, designed primarily based on the classification confidence of the CNN, reflect the generally complementary patterns of the individual classifiers. In consequence, the proposed ensemble classifier MLP-CNN harvests the complementary results acquired from the CNN based on deep spatial feature representation and from the MLP based on spectral discrimination. Meanwhile, limitations of the CNN due to the adoption of convolutional filters such as the uncertainty in object boundary partition and loss of useful fine spatial resolution detail were compensated. The effectiveness of the ensemble MLP-CNN classifier was tested in both urban and rural areas using aerial photography together with an additional satellite sensor dataset. The MLP-CNN classifier achieved promising performance, consistently outperforming the pixel-based MLP, spectral and texturalbased MLP, and the contextual-based CNN in terms of classification accuracy. This research paves the way to effectively address the complicated problem of VFSR image classification.

Very Deep Convolutional Neural Networks for Complex Land Cover Mapping Using Multispectral Remote Sensing Imagery

“…An optimal patch size was determined using a trial-and-error procedure, by taking into account a spatial resolution of 5 m for the input image and the contextual relationship of the objects [41]. In particular, different patch sizes of 5, 10, 15, 20, 25, 30, 35, and 40 were examined, and the patch size of 30 was found to be the optimal value that extracts local spatial correlation within a given neighborhood and contains sufficient information to generate a specific distribution for each object in the image.…”

VPRS-Based Regional Decision Fusion of CNN and MRF Classifications for Very Fine Resolution Remotely Sensed Images

“…Given a CNN classification map, the confidence value of an object is spatially heterogeneous: the central region is often accurately classified, but the boundary region is likely to be misclassified [25]. The two regions (i.e., patch centre and patch boundary) can then be described theoretically by using rough set theory [30].…”

“…The contextual-based CNN classifiers, however, might introduce uncertainties along object boundaries, leading to over-smoothness to some degree [25]. Besides, objects with little spatial information are likely to be misclassified, even for those with distinctive spectral characteristics [25].…”

“…Besides, objects with little spatial information are likely to be misclassified, even for those with distinctive spectral characteristics [25]. In fact, any classifier, regardless of its model structure, predicts with uncertainty when handling spatially and spectrally complex VFSR imagery.…”

“…We previously proposed to create rules to threshold the classification results and deal with uncertainties through decision fusion [25]. This method, although having potential to achieve desirable classification results, involves a large amount of trial and error and prior knowledge of feature characteristics, thus was hard to be generalized and applied in an automatic fashion.…”