Tracking multiple interacting targets in a camera network

Abstract

In this paper we propose a framework for tracking multiple interacting targets in a wide-area camera network consisting of both overlapping and non-overlapping cameras. Our method is motivated from observations that both individuals and groups of targets interact with each other in natural scenes. We associate each raw target trajectory (i.e., a tracklet) with a group state, which indicates if the trajectory belongs to an individual or a group. Structural support vector machine (SSVM) is applied to the group states to decide if merge or split events occur in the scene. Information fusion between multiple overlapping cameras is handled using a homography-based voting scheme. The problem of tracking multiple interacting targets is then converted to a network flow problem, for which the solution can be obtained by the K-shortest paths algorithm. We demonstrate the effectiveness of the proposed algorithm on the challenging VideoWeb dataset in which a large amount of multi-person interaction activities are present. Comparative analysis with state-of-the-art methods is also shown.

Features for Multi-target Multi-camera Tracking and Re-identification

“…Most existing formulations, with some exceptions [10,52,53], are special cases of the multidimensional assignment problem [25]: Input detections are arranged in a graph whose edges encode similarity and whose nodes are then partitioned into identities. Formulations with polynomial time solutions consider evidence along paths of time-consecutive edges [8,16,31,37,38,39,55,75,78] and some build on bipartite matching [12,14,20,26,42,62,71]. Methods that use all pairwise terms, not only timeconsecutive ones, are significantly more accurate but NPhard [18,25,27,28,41,58,61,65,66,67].…”

“…Appearance. Human appearance has been described by color [14,19,20,21,27,32,38,39,42,77,78] and texture descriptors [14,20,26,42,77,78]. Lighting variations are addressed through color normalization [14], exemplarbased approaches [20], or brightness transfer functions learned with [27,38] or without supervision [19,32,77,78].…”

“…Spatial relations between cameras are either explicitly mapped in 3D [19,77], learned by tracking known identities [15,38,39], or obtained by comparing entry/exit rates across pairs of cameras [14,42,48]. Preprocessing methods may fuse data from partially overlapping views [78], while some systems rely on completely overlapping and unobstructed views [1,8,11,33,40]. People entry and exit points may be explicitly modeled on the ground [14,19,42,48] or image plane [32,39].…”

Multi-target Tracking in Multiple Non-overlapping Cameras Using Fast-Constrained Dominant Sets

“…Obviously, learning Brightness Transfer Functions or color correction models requires large amount of training data and they may not be robust against drastic illumination changes across different cameras. Therefore, recent approaches have combined them with spatio-temporal cue which improve multi-target tracking performance [26], [27], [28], [29], [30], [31]. Chen et al [26] utilized human part configurations for every target track from different cameras to describe the across-camera spatio-temporal constraints for across-camera track association, which is formulated as a multi-class classification problem via Markov Random Fields (MRF).…”

GroupSense: A Lightweight Framework for Group Identification

“…The location information may come from various signals for indoor and outdoor localization techniques based on GPS, wireless signals etc. [12], [17], [21], [22], [23], [24]; similarly acoustic information can be extracted from the audio signals captured by the smartphones [25], [26], [27], [28]. We aim to discover the meeting group G [t,t+T ] formed during the period [t, t + T ] from the logged sensor repository X .…”