Real-Time Visual Object Tracking via Few-Shot Learning

Abstract

Visual Object Tracking (VOT) can be seen as an extended task of Few-Shot Learning (FSL). While the concept of FSL is not new in tracking and has been previously applied by prior works, most of them are tailored to fit specific types of FSL algorithms and may sacrifice running speed. In this work, we propose a generalized two-stage framework that is capable of employing a large variety of FSL algorithms while presenting faster adaptation speed. The first stage uses a Siamese Regional Proposal Network to efficiently propose the potential candidates and the second stage reformulates the task of classifying these candidates to a few-shot classification problem. Following such a coarse-to-fine pipeline, the first stage proposes informative sparse samples for the second stage, where a large variety of FSL algorithms can be conducted more conveniently and efficiently. As substantiation of the second stage, we systematically investigate several forms of optimization-based few-shot learners from previous works with different objective functions, optimization methods, or solution space. Beyond that, our framework also entails a direct application of the majority of other FSL algorithms to visual tracking, enabling mutual communication between researchers on these two topics. Extensive experiments on the major benchmarks, VOT2018, OTB2015, NFS, UAV123, Track-ingNet, and GOT-10k are conducted, demonstrating in desirable performance gain and a real-time speed.

Low-shot learning and class imbalance: a survey

“…We note that there are many works which apply "trivial" CI methods (see "Common non-LSL approaches to CI" section) independent of the rest of the model, and thus are not included in the body of this survey nor in Table 1. These measures include loss functions such as focal loss [17][18][19][20][21], class-weighted loss [22][23][24][25], difficulty-weighted loss [19,26], and others [27][28][29][30][31][32]; as well as resampling methods such as random undersampling [33], difficulty-based sampling [34], and other forms of balancing sampling [22,35,36] (citations above are not exhaustive). In addition to the works above, we note but do not include a paper by Li et al which proposes a novel few-shot intent-detection benchmark which contains various forms of class imbalance, but does not propose measures to explicitly counter this imbalance [37].…”

Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey

“…This task closely resembles the FSL task setting, which involves classification using minimal data. Consequently, some researchers [137] have applied FSL to object tracking. However, domain gaps frequently exist between auxiliary data and target data due to variations in devices and data acquisition methods.…”