Multi-Modal Subspace Fusion via Cauchy Multi-Set Canonical Correlations

Abstract

Multi-set canonical correlation analysis (MCCA) is a famous multi-modal coherent subspace learning method. However, sample-based between-modal and within-modal covariance matrices of MCCA usually deviate from real covariance matrices due to noise information and limited sample size. The deviation will weaken the performance of MCCA, especially in image recognition. Aiming at this challenging issue, we correct singular values of sample covariance matrices with the employment of Cauchy estimate theory and further obtain Cauchy covariance matrices that are closer to real covariance matrices. On the basis of Cauchy covariance matrices, we develop a novel multi-modal subspace fusion method, i.e. Cauchy multi-set canonical correlations. By maximizing Cauchy correlations between different modalities and constraining Cauchy scatters of within-modal data, the method can learn a Cauchy coherent fusion subspace with well discriminative power from a few images. Experiment results have shown the effectiveness of the proposed method, promising to the aims of this research.

Exponential Multi-Modal Discriminant Feature Fusion for Small Sample Size

“…More concretely, we decompose the matrix 𝐻 by eigenvalue decomposition [30]: . Inspired by the eigenvalue correction idea [31,32], we construct an exponential discriminative integration matrix 𝐻 ̃ as follows:…”

Robust Pedestrian Detection Based on Multi-Spectral Image Fusion and Convolutional Neural Networks

“…Over the years, a variety of methods for multi-spectral image fusion has been proposed [9,10]. These methods mainly include multi-scale transformation (MDT) [11][12][13], sparse representation [14,15], subspace [16,17], saliency [18,19], and deep networks [20][21][22]. In recent years, deep network-based fusion methods have become a popular topic of research, but these methods are usually based on complex computational models and require a large amount of multi-spectral fusion data.…”