Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis

Volpi, Michele; Camps‐Valls, Gustau; Tuia, Devis

doi:10.1016/j.isprsjprs.2015.02.005

Cited by 125 publications

(71 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The aim is to learn a joint representation, where both (co-registered) images can be compared: this area is especially interesting when methods can align data from multiple sensors (see [132,133]). Three studies employ deep learning to this end:…”

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

Self Cite

2,330

1,204

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version please go to IEEE Geoscience and Remote Sensing Magazine on IEEE XPlore.Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a "black-box" solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization. Following this wave of success and thanks to the increased availability of data and computational resources, the use of deep learning in remote sensing is finally taking off in remote sensing as well. Remote sensing data bring some new challenges for deep learning, since satellite image analysis raises some unique questions that translate into challenging new scientific questions:• Remote sensing data are often multi-modal, e.g. from optical (multi-and hyperspectral) and synthetic aperture radar (SAR) sensors, where both the imaging geometries and the content are completely different. Data and information fusion uses these complementary

show abstract

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

Self Cite

2,330

1,204

View full text Add to dashboard Cite

show abstract

“…This reasoning was brought into multitemporal scenarios, where difference kernels were applied for multitemporal processing or change detection. Extending this reasoning to ensembles of kernels (more than two sources) was possible through multiple kernel learning [85], which was used in multimodal studies for combining spectral and spatial information [86], to combine optical and radar data [76], [87], data form the same satellite but completely different locations [88], or optical data from different satellites [89] in change detection. In all these examples, each data source is used to generate a kernel matrix and all the source-specific kernel matrices are then combined by linear combination into a multimodal similarity matrix.…”

Section: A Feature Level Fusion Through Multiple Kernelsmentioning

confidence: 99%

“…Such setting cannot be handled by standard classification pipelines, unless we proceed by downgrading the WorldView-2 images (e.g., by removing the four nonmatching bands). Recent systems based on multiview analysis [89], [129] can answer this call and be used to transfer model regardless of the dimensionality of the data space, i.e., of the sensor used in the first place.…”

Section: G Cross-sensor Adaptation Via Manifold Alignmentmentioning

confidence: 99%

Multimodal Classification of Remote Sensing Images: A Review and Future Directions

Gómez-Chova¹,

et al. 2015

Self Cite

View full text Add to dashboard Cite

| Earth observation through remote sensing images allows the accurate characterization and identification of materials on the surface from space and airborne platforms. Multiple and heterogeneous image sources can be available for the same geographical region: multispectral, hyperspectral, radar, multitemporal, and multiangular images can today be acquired over a given scene. These sources can be combined/fused to improve classification of the materials on the surface. Even if this type of systems is generally accurate, the field is about to face new challenges: the upcoming constellations of satellite sensors will acquire large amounts of images of different spatial, spectral, angular, and temporal resolutions. In this scenario, multimodal image fusion stands out as the appropriate framework to address these problems. In this paper, we provide a taxonomical view of the field and review the current methodologies for multimodal classification of remote sensing images.We also highlight the most recent advances, which exploit synergies with machine learning and signal processing: sparse methods, kernel-based fusion, Markov modeling, and manifold alignment. Then, we illustrate the different approaches in seven challenging remote sensing applications: 1) multiresolution fusion for multispectral image classification; 2) image downscaling as a form of multitemporal image fusion and multidimensional interpolation among sensors of different spatial, spectral, and temporal resolutions; 3) multiangular image clas, sification; 4) multisensor image fusion exploiting physicallybased feature extractions; 5) multitemporal image classification of land covers in incomplete, inconsistent, and vague image sources; 6) spatiospectral multisensor fusion of optical and radar images for change detection; and 7) cross-sensor adaptation of classifiers. The adoption of these techniques in operational settings will help to monitor our planet from space in the very near future.

show abstract

“…kCCA is the kernel version of CCA [7]. CCA is a multivariate feature extraction method that aims at finding the rotation of two sets of variables that maximizes their joint correlation [32].…”

Section: Kcca Transformation and Nifs Extractionmentioning

confidence: 99%

“…Radiometric normalization can directly make use of the pixel values of an image to establish the corresponding transformation equation for each multi-spectral band in multi-temporal remote sensing data, without the request of any other parameters such as the atmospheric conditions when the remote sensing data obtained [6]. In such a context, radiometric normalization is called spectral alignment [7]. Radiometric normalization builds not only the common radiometric scale/reference but also the radiometric consistency among an image sequence.…”

Section: Introductionmentioning

confidence: 99%

kCCA Transformation-Based Radiometric Normalization of Multi-Temporal Satellite Images

Bai

Tang

2018

Remote Sensing

View full text Add to dashboard Cite

Radiation normalization is an essential pre-processing step for generating high-quality satellite sequence images. However, most radiometric normalization methods are linear, and they cannot eliminate the regular nonlinear spectral differences. Here we introduce the well-established kernel canonical correlation analysis (kCCA) into radiometric normalization for the first time to overcome this problem, which leads to a new kernel method. It can maximally reduce the image differences among multi-temporal images regardless of the imaging conditions and the reflectivity difference. It also perfectly eliminates the impact of nonlinear changes caused by seasonal variation of natural objects. Comparisons with the multivariate alteration detection (CCA-based) normalization and the histogram matching, on Gaofen-1 (GF-1) data, indicate that the kCCA-based normalization can preserve more similarity and better correlation between an image-pair and effectively avoid the color error propagation. The proposed method not only builds the common scale or reference to make the radiometric consistency among GF-1 image sequences, but also highlights the interesting spectral changes while eliminates less interesting spectral changes. Our method enables the application of GF-1 data for change detection, land-use, land-cover change detection etc.

show abstract

Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis

Cited by 125 publications

References 45 publications

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Multimodal Classification of Remote Sensing Images: A Review and Future Directions

kCCA Transformation-Based Radiometric Normalization of Multi-Temporal Satellite Images

Contact Info

Product

Resources

About