Abstract-To improve visual tracking, a large number of papers study more powerful features, or better cue fusion mechanisms, adaptation or contextual models, for instance. A complementary approach consists in improving the track management, that is, deciding when to add a target or stop its tracking, for example in case of failure. This is an essential component for effective multi-object tracking applications, and is often not trivial. Deciding to stop a track or not is a compromise between avoiding erroneous early stopping while tracking is fine, and erroneous continuation of tracking when there is an actual failure. This decision process, very rarely addressed in the literature, is difficult due to, for example, object detector deficiencies or observation models that are insufficient to describe the full variability of tracked objects and deliver reliable likelihood (tracking) information. This paper addresses the track management issue and presents a real-time, online multi-face tracking algorithm that effectively deals with the above difficulties. The tracking itself is formulated in a multiobject state-space Bayesian filtering framework solved with Markov Chain Monte Carlo. Within this framework, an explicit probabilistic filtering step decides when to add or remove a target from the tracker, where decisions rely on multiple cues such as face detections, likelihood measures, long term observations, and track state characteristics. The method has been applied to three challenging datasets of more than 9 hours in total, and demonstrate a significant performance increase compared to more traditional approaches (MCMC, RJ-MCMC) only relying on head detections and likelihoods for track management.