Fast CNN-Based Object Tracking Using Localization Layers and Deep Features Interpolation

Abstract

Object trackers based on Convolution Neural Network (CNN) have achieved state-of-the-art performance on recent tracking benchmarks, while they suffer from slow computational speed. The high computational load arises from the extraction of the feature maps of the candidate and training patches in every video frame. The candidate and training patches are typically placed randomly around the previous target location and the estimated target location respectively. In this paper, we propose novel schemes to speedup the processing of the CNN-based trackers. We input the whole region-of-interest once to the CNN to eliminate the redundant computations of the random candidate patches. In addition to classifying each candidate patch as an object or background, we adapt the CNN to classify the target location inside the object patches as a coarse localization step, and we employ bilinear interpolation for the CNN feature maps as a fine localization step. Moreover, bilinear interpolation is exploited to generate CNN feature maps of the training patches without actually forwarding the training patches through the network which achieves a significant reduction of the required computations. Our tracker does not rely on offline video training. It achieves competitive performance results on the OTB benchmark with 8x speed improvements compared to the equivalent tracker.

Towards an Efficient Hardware Implementation of CNN-Based Object Trackers

“…Accordingly, the training patches are not forwarded through the convolutional layers, and hence, a significant reduction in the processing time is achieved. As reported in [3], a comparable performance with the baseline tracker is achieved for all the tracking challenges, while obtaining a speed improvement of 8.8x on the same CPU. Fig.…”

“…The main idea of ILNET [3], however, is to reduce the number of the convolutional computations in the network, and hence, achieve a processing speed-up factor. The whole Region of Interest (ROI) is forwarded to the network instead of small random patches and a 15x15x512 feature map is obtained.…”

“…1. ILENT network structure [3] and short-period respectively as proposed in [2]. This longperiod and short-period in both MDNET and ILNET are 100 and 20 frames respectively.…”

“…Reference [3] introduced a CNN-based object tracker named Interpolation and Localization Network (ILNET) where the speed limitation of the CNN-based object trackers is addressed. In ILNET, bi-linear interpolation and linear interpolation schemes are employed on the translation domain and on the scale domain respectively.…”

Multiple Vehicles and People Tracking in Aerial Imagery Using Stack of Micro Single-Object-Tracking CNNS

“…In recent years, the VOT methods based on deep learning specifically Convolutional Neural Networks (CNNs) (Girshick et al, 2014, Girshick, 2015, Ren et al, 2015, Lin et al, 2017 have shown promising performances in MOT scenarios (Wojke et al, 2017, Bewley et al, 2016. However, most of these methods suffer from high computational costs and slow processing, especially extracting features from each candidate object locations in every frame (El-Shafie et al, 2019). The complexity increases exponentially by increasing the number of objects.…”