Learning Multi-domain Convolutional Neural Networks for Visual Tracking

Abstract

We propose a novel visual tracking algorithm based on the representations from a discriminatively trained Convolutional Neural Network (CNN). Our algorithm pretrains a CNN using a large set of videos with tracking groundtruths to obtain a generic target representation. Our network is composed of shared layers and multiple branches of domain-specific layers, where domains correspond to individual training sequences and each branch is responsible for binary classification to identify the target in each domain. We train the network with respect to each domain iteratively to obtain generic target representations in the shared layers. When tracking a target in a new sequence, we construct a new network by combining the shared layers in the pretrained CNN with a new binary classification layer, which is updated online. Online tracking is performed by evaluating the candidate windows randomly sampled around the previous target state. The proposed algorithm illustrates outstanding performance compared with stateof-the-art methods in existing tracking benchmarks.

Tracking by Natural Language Specification

“…Wang et al [36] transform deep network optimization into sequential ensemble learning by online training. In [28] Nam et al enable fine-tuning to more than one domain, where each domain is represented by a single training sequence. In [34,3], tracking is cast as instance search for which a Siamese network architecture is used.…”

Learning to Track at 100 FPS with Deep Regression Networks

“…Some researchers have also attempted to use neural networks for tracking within the traditional online training framework [26,27,34,37,35,30,39,7,24,16], showing state-of-the-art results [30,7,21]. Unfortunately, neural networks are very slow to train, and if online training is required, then the resulting tracker will be very slow at test time.…”

“…Unfortunately, neural networks are very slow to train, and if online training is required, then the resulting tracker will be very slow at test time. Such trackers range from 0.8 fps [26] to 15 fps [37], with the top performing neural-network trackers running at 1 fps on a GPU [30,7,21]. Hence, these trackers are not usable for most practical applications.…”

“…In contrast, our tracker is trained offline to learn a generic relationship between appearance and motion, so no online training is required. Second, most trackers take a classification-based approach, classifying many image patches to find the target object [26,27,37,30,39,24,33]. In contrast, our tracker uses a regression-based approach, requiring just a single feed-forward pass through the network to regresses directly to the location of the target object.…”

“…Our real-time speed is due to two factors. First, most previous neural network trackers are trained online [26,27,34,37,35,30,39,7,24]; however, training neural networks is a slow process, leading to slow tracking. In contrast, our tracker is trained offline to learn a generic relationship between appearance and motion, so no online training is required.…”