Robust People Tracking with Global Trajectory Optimization

Abstract

Given three or four synchronized videos taken at eye level and from different angles, we show that we can effectively use dynamic programming to accurately follow up to six individuals across thousands of frames in spite of significant occlusions. In addition, we also derive metrically accurate trajectories for each one of them. Our main contribution is to show that multi-person tracking can be reliably achieved by processing individual trajectories separately over long sequences, provided that a reasonable heuristic is used to rank these individuals and avoid confusing them with one another. In this way, we achieve robustness by finding optimal trajectories over many frames while avoiding the combinatorial explosion that would result from simultaneously dealing with all the individuals.

Exploiting Pedestrian Interaction via Global Optimization and Social Behaviors

“…Several methods can be used such as Markov Chain Monte Carlo (MCMC) [13], multi-level Hungarian [14], inference in Bayesian networks [15] or the Nash Equilibrium of game theory [16]. In [17] an efficient approximative Dynamic Programming (DP) scheme is presented, in which trajectories are estimated one after the other. This means that if a trajectory is formed using a certain detection, the other trajectories which are computed later will not be able to use that detection anymore.…”

“…Using the information of the Probability Occupancy Map, the problem is formulated either as a max-flow and solved with Simplex, or as a min-cost and solved using k-shortest paths, which is a more efficient solution. Both methods show a far superior performance when compared to the same approach with DP [17]. The authors of [19] also define the problem as a maximum flow on an hexagonal grid, but instead of using matching individual detections, they make use of tracklets.…”

Coupled Detection and Trajectory Estimation for Multi-Object Tracking

“…While many approaches rely on background subtraction from a static camera for the former (e.g. [20,12,2]), several recent approaches have started to explore the possibilities of combining tracking with detection [17,1,8,22]. This has been helped by the astonishing progress object detection research has made over the last few years [4,14,16,21], which has resulted in state-of-the-art detectors that are applicable in complex outdoor scenes.…”

“…Our approach implements a feedback loop, which passes on predicted object locations as a prior to influence detection, while at the same time choosing between and reevaluating trajectory hypotheses in the light of new evidence. In contrast to previous approaches, which optimize individual trajectories in a temporal window [2,23] or over sensor gaps [11], our approach tries to find a globally optimal combined solution for all detections and trajectories, while incorporating physical constraints such that no two objects can occupy the same physical space, nor explain the same image pixels at the same time. The task complexity is reduced by only selecting between a limited set of plausible hypotheses, which makes the approach computationally feasible.…”

Multi-Person Bayesian Tracking with Multiple Cameras

“…Some authors are using a single-object state-space model [16,17], where the modes of the state are identified as individual objects. Fleuret et al [18] use a greedy approach, by extracting the different tracks one by one from instantenous object detection features using a Hidden Markov model (HMM), and removing the detection features associated with the already extracted tracks. However, only a rigourous formulation of the MOT problem using a multi-object state space allows to formalize in a principled way the different components that one may wish for a tracker: uniquely identifying targets, modeling their interactions, handling the variability of the number of objects using track birth and death mechanisms.…”