Evolutionary particle filtering for sequential dependency learning from video data

Abstract

We describe a novel learning scheme for hidden dependencies in video streams. The proposed scheme aims to transform a given sequential stream into a dependency structure of particle populations. Each particle population summarizes an associated segment. The novel point of the proposed scheme is that both of dependency learning and segment summarization are performed in an unsupervised online manner without assuming priors. The proposed scheme is executed in two-stage learning. At the first stage, a segment corresponding to a common dominant image is estimated using evolutionary particle filtering. Each dominant image is depicted based on combinations of image descriptors. Prevailing features of a dominant image are selected through evolution. Genetic operators introduce the essential diversity preventing sample impoverishment. At the second stage, transitional probability between the estimated segments is computed and stored. The proposed scheme is applied to extract dependencies in an episode of a TV drama. We demonstrate performance by comparing to human estimations.