Learning Slow Features for Behaviour Analysis

Abstract

A recently introduced latent feature learning technique for time varying dynamic phenomena analysis is the socalled Slow Feature Analysis (SFA). SFA is a deterministic component analysis technique for multi-dimensional sequences that by minimizing the variance of the first order time derivative approximation of the input signal finds uncorrelated projections that extract slowly-varying features ordered by their temporal consistency and constancy. In this paper, we propose a number of extensions in both the deterministic and the probabilistic SFA optimization frameworks. In particular, we derive a novel deterministic SFA algorithm that is able to identify linear projections that extract the common slowest varying features of two or more sequences. In addition, we propose an Expectation Maximization (EM) algorithm to perform inference in a probabilistic formulation of SFA and similarly extend it in order to handle two and more time varying data sequences. Moreover, we demonstrate that the probabilistic SFA (EM-SFA) algorithm that discovers the common slowest varying latent space of multiple sequences can be combined with dynamic time warping techniques for robust sequence timealignment. The proposed SFA algorithms were applied for facial behavior analysis demonstrating their usefulness and appropriateness for this task.

Joint Unsupervised Face Alignment and Behaviour Analysis

“…The gold standard in unsupervised behaviour analysis is (a) to track facial landmark points and (b) use their motion to perform analysis. For example in [2,10] person specific trackers were used, which require manual annotation, and in [15,3] a generic tracker was employed followed by a manual correction step. The goals of the experiments are two fold: (1) to show that the method can correctly track landmarks from a crude face detector and (2) to show that the extracted features can represent the dynamics of the behaviour.…”

“…Local Binary Patterns (LBPs) and SIFT features ( [13], [14]). Finally, when it comes to temporal alignment of facial events, the tracked facial landmarks are aligned after being tracked [3,15,16], usually by the application of a person specific tracker.…”

Slow features nonnegative matrix factorization for temporal data decomposition

“…The temporal slowness learning principle in SFA was motivated by the empirical observation that higher order meanings of sensory data, such as objects and their attributes, are often more persistent (i.e., change smoothly) than the independent activation of any single sensory receptor. For instance, in facial behaviour analysis it has been recently shown that SFA learning can discover mapping functions between an input image sequence that varies quickly and the corresponding high-level semantic concepts that vary slowly [4].…”

Joint Unsupervised Deformable Spatio-Temporal Alignment of Sequences

“…For example, the typical framework for temporal alignment of two sequences displaying objects that undergo non-rigid deformations, e.g. a facial expression, is the following [34,35,19,30,17]:…”

“…Temporal alignment is the first step towards analysis and synthesis of human and animal motion, temporal clustering of sequences and behaviour segmentation [34,14,35,19,30,17,36]. Spatial image alignment is among the main computer vision topics [3,16,1].…”