Multilinear subspace analysis of image ensembles

Abstract

Multilinear algebra, the algebra of higher-order tensors, offers a potent mathematical framework for analyzing ensembles of images resulting from the interaction of any number of underlying factors. We present a dimensionality reduction algorithm that enables subspace analysis within the multilinear framework. This N -mode orthogonal iteration algorithm is based on a tensor decomposition known as the N -mode SVD, the natural extension to tensors of the conventional matrix singular value decomposition (SVD). We demonstrate the power of multilinear subspace analysis in the context of facial image ensembles, where the relevant factors include different faces, expressions, viewpoints, and illuminations. In prior work we showed that our multilinear representation, called TensorFaces, yields superior facial recognition rates relative to standard, linear (PCA/eigenfaces) approaches. Here, we demonstrate factor-specific dimensionality reduction of facial image ensembles. For example, we can suppress illumination effects (shadows, highlights) while preserving detailed facial features, yielding a low perceptual error.

Bilinear Active Appearance Models

“…Recent work in Multi-linear models (e.g. [35,39,40]) provide the basis to incorporate multifactorial models that can decouple the image changes due to expression, identity or pose. In particular, pose changes are usually hard to represent with 2D linear models because of the resulting non-linear point transformations (e.g.…”

Homeomorphic Manifold Analysis: Learning Decomposable Generative Models for Human Motion Analysis

“…where ψ(x) is a nonlinear kernel map from a representation of the body configuration to a kernel induced space and each a i is a vector representing a parameterization of orthogonal factor i, C is a core tensor, × i is mode-i tensor product as defined in [22,23]. The model in equation 2 is a generalization over the model introduced in [13] where only one factor can be decomposed.…”

“…This can be achieved by applying higher-order orthogonal iteration for dimensionality reduction [28,23]. Final subspace representation is…”

“…Given such mapping the decomposition in equation 1 can be achieved using tensor analysis of the coefficient space such that the coefficient B a are obtained from a multilinear [23] model…”

“…Singular value decomposition (SVD) can be used for PCA analysis and iterative SVD with vector transpose for bilinear analysis. Higher-order tensor analysis can be achieved by higher-order sigular value decomposition (HOSVD) with unfolding, which is a generalization of SVD [22,23,27].…”