Nonlinear modeling of scattered multivariate data and its application to shape change

Abstract

We are given a set of points in a space of high dimension. For instance, this set may represent many visual appearances of an object, a face or a hand. We address the problem of approximating this set by a manifold in order to have a compact representation of the object appearance. When the scattering of this set is approximately an ellipsoid, then the problem has a well-known solution given by Principal Components Analysis (PCA). However, in some situations like object displacement learning or face learning this linear technique may be ill-adapted and nonlinear approximation has to be introduced. The method we propose can be seen as a Non Linear PCA (NLPCA), the main difficulty being that the data are not ordered. We propose an index which favours the choice of axes preserving the neighborhood of the nearest neighbours. These axes determine an order for visiting all the points when smoothing. Finally a new criterion, called "generalization error", is introduced to determine the smoothing rate, that is the knot number of the spline fitting. Experimental results conclude this paper: the method is tested on artificial data and on two data bases used in visual learning.

Representing Complex Data Using Localized Principal Components with Application to Astronomical Data

“…While most NLPCA algorithms work by applying some nonlinear, but parametric, transformation in the projection and/or the reconstruction step ( [47], [8], among others), Bolten et al [5] allowed for a nonparametric reconstruction g. Using in the projection step a nonlinear transformation followed by a linear mapping, i.e. f (x) = W φ(x), φ : R p → R l , W ∈ R d×l , the reconstructed curve takes the form g(W φ(x)), which is estimated using projection pursuit regression [23].…”

“…The data comprise several thousand spectra showing variance in the three astrophysical parameters temperature (in Kelvin), metallicity and gravity; the latter two variables are on a logarithmic scale. 8 Temperature is a "strong" parameter, meaning it accounts for most of the variance across the data set. Gravity and metallicity, in contrast, are weak.…”

A nonlinear PCA based on manifold approximation

“…This makes any descent method useless for its maximization. To overcome this problem, we developped a simulated annealing algorithm described in [2,3]. This method ensures to reach the global maximum of the index during the step (8) of the algorithm [8].…”

Intrinsic dimension estimation by maximum likelihood in isotropic probabilistic PCA

“…Examples of such approaches include PCA [18] sometimes associated with Cattell's scree test [12] and its non linear extensions based either on auto-associative models [19,13] or Mercer kernels [29]. Multidimensional scaling type algorithms aim at finding the projection which (locally) preserve the distances among data.…”