No abstract

Multilinear algebra, the algebra of higher-order tensors, offers a potent mathematical framework for analyzing ensembles of images resulting from the interaction of any number of underlying factors. We present a dimensionality reduction algorithm that enables subspace analysis within the multilinear framework. This N -mode orthogonal iteration algorithm is based on a tensor decomposition known as the N -mode SVD, the natural extension to tensors of the conventional matrix singular value decomposition (SVD). We demonstrate the power of multilinear subspace analysis in the context of facial image ensembles, where the relevant factors include different faces, expressions, viewpoints, and illuminations. In prior work we showed that our multilinear representation, called TensorFaces, yields superior facial recognition rates relative to standard, linear (PCA/eigenfaces) approaches. Here, we demonstrate factor-specific dimensionality reduction of facial image ensembles. For example, we can suppress illumination effects (shadows, highlights) while preserving detailed facial features, yielding a low perceptual error.

Natural images are rhe composite conseqirence of mulriple facrors related to scene structure, illumination, and imaging. For facial images, the factors include different facial geometries, expressions, heud poses, and lighting conditi[~ns. We upply multilinear algebra, rhe algebra of higherorder tensors, to obrain a parsimonious representation of facial image ensembles which separates these facfors. Our represenratioti, called TensorFaces, yields impmved facial recognirion rues relative to standard eigenfaces.

Independent Components Analysis (ICA) maximizes the statistical independence of the representational components of a training image ensemble, but it cannot distinguish between the different factors, or modes, inherent to image formation, including scene structure, illumination, and imaging. We introduce a nonlinear, multifactor model that generalizes ICA. Our Multilinear ICA (MICA) model of image ensembles learns the statistically independent components of multiple factors. Whereas ICA employs linear (matrix) algebra, MICA exploits multilinear (tensor) algebra. We furthermore introduce a multilinear projection algorithm which projects an unlabeled test image into the N constituent mode spaces to simultaneously infer its mode labels. In the context of facial image ensembles, where the mode labels are person, viewpoint, illumination, expression, etc., we demonstrate that the statistical regularities learned by MICA capture information that, in conjunction with our multilinear projection algorithm, improves automatic face recognition.

Abstract

Abstract

Numerical multilinear (tensor) algebra is a principled mathematical approach to disentangling and explicitly and parsimoniously representing the essential factors or modes of image formation, among them illumination, scene geometry, and imaging, thereby dramatically improving the performance of appearance-based recognition. Generalizing concepts from linear (matrix) algebra, we define the identity tensor and the pseudo-inverse tensor and we employ them to develop a multilinear projection algorithm, which is natural for performing recognition in the tensor algebraic framework. Our multilinear projection algorithm simultaneously projects an unlabeled test image into multiple constituent mode spaces spanned by learned, mode-specific basis sets in order to infer its mode labels. Multilinear projection is applied to unconstrained facial image recognition, where the mode labels are person identity, viewpoint, illumination, etc.

Adaptive meshes an? dynamic networks of nodal masses interconnected by adjustable springs. They are useful for nonuniformly sampling and reconstructing visual data. This paper extends the adaptive mesh model i n the following ways: it (i) develops open adaptive meshes and closed adaptive shells based on triangular and rectangular elements, (ii) proposes a discontinuity detection and preservation algorithm suitable for the model, and (iii) develops techniques for adaptive hierarchical subdivision of adaptive meshes and shells. The eztended model is applied to image and 30 surface data.