Learning a Robust Society of Tracking Parts Using Co-occurrence Constraints

Abstract

Object tracking is an essential problem in computer vision that has been researched for several decades. One of the main challenges in tracking is to adapt to object appearance changes over time and avoiding drifting to background clutter. We address this challenge by proposing a deep neural network composed of different parts, which functions as a society of tracking parts. They work in conjunction according to a certain policy and learn from each other in a robust manner, using cooccurrence constraints that ensure robust inference and learning. From a structural point of view, our network is composed of two main pathways. One pathway is more conservative. It carefully monitors a large set of simple tracker parts learned as linear filters over deep feature activation maps. It assigns the parts different roles. It promotes the reliable ones and removes the inconsistent ones. We learn these filters simultaneously in an efficient way, with a single closed-form formulation, for which we propose novel theoretical properties. The second pathway is more progressive. It is learned completely online and thus it is able to better model object appearance changes. In order to adapt in a robust manner, it is learned only on highly confident frames, which are decided using co-occurrences with the first pathway. Thus, our system has the full benefit of two main approaches in tracking. The larger set of simpler filter parts offers robustness, while the full deep network learned online provides adaptability to change. As shown in the experimental section, our approach achieves state of the art performance on the challenging VOT17 benchmark, outperforming the published methods both on the general EAO metric and in the number of fails, by a significant margin.

Adaptive convolutional layer selection based on historical retrospect for visual tracking

“…VOT‐2017 dataset : In the more important and difficult benchmark, VOT‐2017 [25], several trackers are compared with our tracker, these trackers include the very top state‐of‐the‐art methods: STP [42], CFWCR [43], convolutional features for correlation filters (CFCF) [57], ECO [45], CCOT [44] and other recent methods: RCPF [58], unified convolutional tracker (UCT) [59], SPCT [60], SiamFC [33], Staple [1] and DPT [49].…”

“…VOT-2016 dataset: We compare our tracker with 22 state-ofthe-art trackers on the VOT-2016 benchmark including society of tracking parts (STP) [42], CFWCR [43], ECO [45], CCOT [44], tree-structured convolutional neural network (TCNN) [48], SSAT [24], DPT [49], SiamFC [33], deepMKCF [50], new scale adaptive and multiple feature (NSAMF) [51], colour-aware complex cell tracker (CCCT) [52], structure output deep learning tracker (SO-DLT) [31], HCF [20], DAT [53], scale adaptive mean-shift (ASMS) [54], KCF [4], SAMF [51], DSST [3], tracking with Gaussian processes regression (TGPR) [55], multiple instance learning (MIL) [16], structured output tracking with kernels (STRUCK) [17] and incremental learning for visual tracking (IVT) [56]. Table 2 shows the results of our tracker and other trackers.…”

Graph-based topic models for trajectory clustering in crowd videos

“…Using the orientation and magnitude of extracted tracklets, one-dimensional descriptors were derived and fed into one-class support vector machine (SVM) classifier for abnormality detection. Recently, Burceanu and Leordeanu [9] proposed a neural network object tracker with two pathways; the FilterParts and the ConvNetPart. The first pathway is robust to background noises while the second one is robust to object appearance changes over time.…”

Deep Learning for Visual Tracking: A Comprehensive Survey

“…• Robust target representation: Providing a powerful target representation is the main advantage of employing CNNs for visual tracking. To achieve the goal of learning generic representations for target modeling and constructing a more robust target models, the main contributions of methods are classified into: i) offline training of CNNs on large-scale datasets for visual tracking [63], [68], [80], [89], [97], [100], [101], [104], [112], [116], [135], [137], [142], [144], [153], [165], [168], [169], [173], ii) designing specific deep convolutional networks instead of employing pre-trained models [63], [68], [70], [72], [73], [75], [76], [80], [82], [89], [97], [100], [101], [104], [105], [108], [112], [116], [127], [135], [137], [141], [142], [144], [146], [150],…”