Generalizations of principal component analysis, optimization problems, and neural networks

In this paper, we adopt a differential-geometry viewpoint to tackle the problem of learning a distance online. As this problem can be cast into the estimation of a fixed-rank positive semidefinite (PSD) matrix, we develop algorithms that exploits the rich geometry structure of the set of fixed-rank PSD matrices. We propose a method which separately updates the subspace of the matrix and its projection onto that subspace. A proper weighting of the two iterations enables to continuously interpolate between the problem of learning a subspace and learning a distance when the subspace is fixed.Index Terms-Kernel and metric learning, low-rank approximation, online learning, manifold-based optimization.

show abstract

“…with U ∈ St(n, r) that spans the subspace of interest [13,14]. Note that (16) remains invariant by the transformation U → U O with O ∈ O(r).…”

Section: Connection With Subspace Learningmentioning

confidence: 99%

From subspace learning to distance learning: A geometrical optimization approach

Meyer

Journée

Bonnabel

et al. 2009

2009 IEEE/SP 15th Workshop on Statistical Signal Processing

View full text Add to dashboard Cite

show abstract

“…PCA provides an optimal solution to several signal representation problems, making it useful for feature extraction and dimensionality reduction in a wide range of applications [8], [7]. Its main drawback is that it assumes a Gaussian distribution of the data.…”

Section: An Affine Invariant Function Using Pca Basesmentioning

confidence: 99%

An Affine Invariant Function using PCA Bases with an Application to Within-Class Object Recognition

Tzimiropoulos

Mitianoudis

Stathaki

2007

2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07

View full text Add to dashboard Cite

The problem of shape-based recognition of objects under affine transformations is considered. We focus on the construction of a robust and highly discriminative affine invariant function that can be used for within-class object recognition applications. Using the boundaries of the objects of interest, a training scheme, based on Principal Component Analysis (PCA), is proposed to derive a set of basis functions with desired properties. The derived bases are then used for the construction of a novel affine invariant function. The proposed invariant function is evaluated for the problem of aircraft silhouette identification and appears to achieve comparable performance to a popular wavelet-based affine invariant function. At the same time, the proposed framework is much simpler than that based on wavelet analysis.

show abstract

“…The extension to nonlinear PCA (NLPCA) is not unique, due to the lack of a unified mathematical structure and an efficient and reliable algorithm, and in some cases due to excessive freedom in selection of representative basis functions [51,28]. Several methods have been proposed for nonlinear PCA such as, the five-layer feedforward associative network [41] and the kernel PCA [66].…”

Section: Nonlinear Pca and Principal Manifoldsmentioning

confidence: 99%

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Yin

2008

Lecture Notes in Computational Science and Enginee

View full text Add to dashboard Cite

Summary. This chapter provides an overview on the self-organised map (SOM) in the context of manifold mapping. It first reviews the background of the SOM and issues on its cost function and topology measures. Then its variant, the visualisation induced SOM (ViSOM) proposed for preserving local metric on the map, is introduced and reviewed for data visualisation. The relationships among the SOM, ViSOM, multidimensional scaling, and principal curves are analysed and discussed. Both the SOM and ViSOM produce a scaling and dimension-reduction mapping or manifold of the input space. The SOM is shown to be a qualitative scaling method, while the ViSOM is a metric scaling and approximates a discrete principal curve/surface. Examples and applications of extracting data manifolds using SOM-based techniques are presented.Key words: Self-organising maps, principal curve and surface, data visualisation, topographic mapping IntroductionFor many years, artificial neural networks have been studied and used to construct information processing systems based on or inspired by natural biological neural structures. They not only provide solutions with improved performance when compared with traditional problem-solving methods, but also give a deeper understanding of human cognitive abilities. Among the various existing neural network architectures and learning algorithms, Kohonen's self-organising map (SOM) [35] is one of most popular neural network models. Developed for an associative memory model, it is an unsupervised learning algorithm with simple structures and computational forms, and is motivated by the retina-cortex mapping. Self-organisation in general is a fundamental pattern recognition process, in which intrinsic inter-and intra-pattern relationships within the data set are learnt without the presence of a potentially biased or subjective external influence. The SOM can provide topologically

show abstract

Generalizations of principal component analysis, optimization problems, and neural networks

Cited by 230 publications

References 16 publications

From subspace learning to distance learning: A geometrical optimization approach

From subspace learning to distance learning: A geometrical optimization approach

An Affine Invariant Function using PCA Bases with an Application to Within-Class Object Recognition

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Contact Info

Product

Resources

About