Semi-supervised Laplacian Regularization of Kernel Canonical Correlation Analysis

Abstract

Kernel canonical correlation analysis (KCCA) is a dimensionality reduction technique for paired data. By finding directions that maximize correlation, KCCA learns representations that are more closely tied to the underlying semantics of the data rather than noise. However, meaningful directions are not only those that have high correlation to another modality, but also those that capture the manifold structure of the data. We propose a method that is simultaneously able to find highly correlated directions that are also located on high variance directions along the data manifold. This is achieved by the use of semi-supervised Laplacian regularization of KCCA. We show experimentally that Laplacian regularized training improves class separation over KCCA with only Tikhonov regularization, while causing no degradation in the correlation between modalities. We propose a model selection criterion based on the Hilbert-Schmidt norm of the semi-supervised Laplacian regularized cross-covariance operator, which we compute in closed form.

Joint Unsupervised Deformable Spatio-Temporal Alignment of Sequences

“…The above problem is a kind of temporally regularized orthogonal CCA.Temporal regularisation is probably the reason that the proposed approach outperforms CTW (which does not employ any temporal regularisation). Even though Laplacian regularization of component analysis techniques has recently been significantly studied [7], Laplacian regularization for CCA models has not received much attention [5]. To the best of our knowledge, this is the first component analysis methodology which can 1 …”

Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification

“…3) Graph-based: some recent advances adopt Gaussian Processes [21,17] or Markov Random Walks [2]. Transduction by Laplacian graph [4,10] is also shown to be able to solve multi-class semi-supervised problems; although these algorithms make use of the relationship between unlabeled and labeled data, their computational complexity is demanding, e.g. of O(n 3 ).…”

Leveraging Social Bookmarks from Partially Tagged Corpus for Improved Web Page Clustering

“…In another work, Blaschko et al [Blaschko and Lampert 2008] use CCA for clustering images using the associated text as a second view. Both of these works assume that the views are complete, unlike the setting we considered in this paper.…”

“…For example, one can take a probabilistic approach to CCA [Rai and Daumé III 2009] and treat the missing tags for non-tagged webpages as latent variables. In the non-probabilistic setting, one can use the semi-supervised variants of CCA [Blaschko et al 2008;Kim and Pavlovic 2009] which do not require full information from both the views. Alternatively, a somewhat similar way of accomplishing this would be to write a combined eigenvalue problem with one part of it being CCA on the tagged webpages, and the other being LSA on the non-tagged webpages.…”