Inter-camera Association of Multi-target Tracks by On-Line Learned Appearance Affinity Models

Abstract

Abstract. We propose a novel system for associating multi-target tracks across multiple non-overlapping cameras by an on-line learned discriminative appearance affinity model. Collecting reliable training samples is a major challenge in on-line learning since supervised correspondence is not available at runtime. To alleviate the inevitable ambiguities in these samples, Multiple Instance Learning (MIL) is applied to learn an appearance affinity model which effectively combines three complementary image descriptors and their corresponding similarity measurements. Based on the spatial-temporal information and the proposed appearance affinity model, we present an improved inter-camera track association framework to solve the "target handover" problem across cameras. Our evaluations indicate that our method have higher discrimination between different targets than previous methods.

Person re-identification in crowd

“…Color is the most commonly used appearance feature encoded in the form of either histograms (Gheissari et al, 2006;Gray and Tao, 2008;Javed et al, 2008;Oliveira and Luiz, 2009;Cheng et al, 2009;Farenzena et al, 2010;Prosser et al, 2010;Kuo et al, 2010;Zheng et al, 2011) or cumulative histograms (Berdugo et al, 2010), which are simple to compute and scale invariant. Different color channels and their combination can be used: the Hue channel from the HSV color space (Oliveira and Luiz, 2009); the Hue and Saturation channels jointly (Gheissari et al, 2006); or the three channels of the HSV color space (Farenzena et al, 2010).…”

“…Javed et al (Javed et al, 2008) presented a video-based dataset with three sequences composed of up to three cameras from indoor and outdoor scenarios with large illumination changes and up to four fully visible people. Finally, a more challenging dataset in terms of occlusions is that with three outdoor cameras where up to ten people walk alone or in small groups (Kuo et al, 2010).…”

“…The distribution of spatial patches can be directly extracted in the frequency domain where Discrete Cosine Transform (DCT) coefficients can be used as textural features (Bauml et al, 2010). Finally, spatial patch distribution can be extracted by computing the first and the second derivatives of the person patch resulting in a covariance matrix (Wang et al, 2007;Kuo et al, 2010). Symmetric regions of a patch can be an alternative to the covariance matrix.…”

“…In particular, Gabor and Schmid filters, and HAAR-like features are local descriptors suitable for small patches, while the ratios, RHSP, salient edgels, DCT coefficients, and covariance matrix can only be applied to people patches at medium/high resolution. Furthermore, a Histogram of Oriented Gradients (HOG) gives information on the orientation of the edges in the patch (Wang et al, 2007;Kuo et al, 2010), creating a feature that models the shape of the object by its edge distribution. However, HOG features are only invariant to changes in illumination and not to changes in pose and scale.…”

“…A spatio-temporal over-segmentation of patches over 10 frames can be used to create a signature for each person (Gheissari et al, 2006). However, the most common approach is to keep all the available features extracted from single patches over time and then perform association by analyzing the similarity among all the available features (Javed et al, 2008;Berdugo et al, 2010;Kuo et al, 2010). Features can also be incrementally updated over time, for example using Incremental MCSHR (IMCSHR) that updates MCSHR in order to increase robustness to abrupt changes in illumination (Madden et al, 2007).…”