ÐWe describe new techniques to detect and analyze periodic motion as seen from both a static and a moving camera. By tracking objects of interest, we compute an object's self-similarity as it evolves in time. For periodic motion, the self-similarity measure is also periodic and we apply Time-Frequency analysis to detect and characterize the periodic motion. The periodicity is also analyzed robustly using the 2D lattice structures inherent in similarity matrices. A real-time system has been implemented to track and classify objects using periodicity. Examples of object classification (people, running dogs, vehicles), person counting, and nonstationary periodicity are provided.

No abstract

Abstract. We present a novel technique for motion-based recognition of individual gaits in monocular sequences. Recent work has suggested that the image self-similarity plot of a moving person/object is a projection of its planar dynamics. Hence we expect that these plots encode much information about gait motion patterns, and that they can serve as good discriminants between gaits of different people. We propose a method for gait recognition that uses similarity plots the same way that face images are used in eigenface-based face recognition techniques. Specifically, we first apply Principal Component Analysis (PCA) to a set of training similarity plots, mapping them to a lower dimensional space that contains less unwanted variation and offers better separability of the data. Recognition of a new gait is then done via standard pattern classification of its corresponding similarity plot within this simpler space. We use the k-nearest neighbor rule and the Euclidian distance. We test this method on a data set of 40 sequences of six different walking subjects, at 30 FPS each. We use the leave-one-out crossvalidation technique to obtain an unbiased estimate of the recognition rate of 93%.

No abstract

Gait is one of the few biometrics that can be measured at a distance, and is hence useful for passive surveillance as well as biometric applications. Gait recognition research is still at its infancy, however, and we have yet to solve the fundamental issue of finding gait features which at once have sufficient discrimination power and can be extracted robustly and accurately from low-resolution video. This paper describes a novel gait recognition technique based on the image self-similarity of a walking person. We contend that the similarity plot encodes a projection of gait dynamics. It is also correspondence-free, robust to segmentation noise, and works well with low-resolution video. The method is tested on multiple data sets of varying sizes and degrees of difficulty. Performance is best for fronto-parallel viewpoints, whereby a recognition rate of 98% is achieved for a data set of 6 people, and 70% for a data set of 54 people

No abstract

No abstract

The visual motion of the mouth and the corresponding audio data generated when a person speaks are highly correlated. This fact has been exploited for liphpeechreading and for improving speech recognition. We describe a method of automatically detecting a talking person (both spatially and temporally) using video and audio data from a single microphone. The audio-visual correlation is learned using a time delayed neural network, which is then used to perform a spatio-temporal search for a speaking person. Applications include video conferencing, video indexing, and improving human computer interaction (HCI). An example HCI application is provided.