Re$^3$: Re al-Time Recurrent Regression Networks for Visual Tracking of Generic Objects

Abstract

Robust object tracking requires knowledge and understanding of the object being tracked: its appearance, its motion, and how it changes over time. A tracker must be able to modify its underlying model and adapt to new observations. We present Re 3 , a real-time deep object tracker capable of incorporating temporal information into its model. Rather than focusing on a limited set of objects or training a model at testtime to track a specific instance, we pretrain our generic tracker on a large variety of objects and efficiently update on the fly; Re 3 simultaneously tracks and updates the appearance model with a single forward pass. This lightweight model is capable of tracking objects at 150 FPS, while attaining competitive results on challenging benchmarks. We also show that our method handles temporary occlusion better than other comparable trackers using experiments that directly measure performance on sequences with occlusion.

Attention‐aware fully convolutional neural network with convolutional long short‐term memory network for ultrasound‐based motion tracking

“…Both the training and testing data involved 64 2D+t sequences; herein, the training size is 25, while the testing size is 39. The data had varying spatial (0.27-0.77 mm) and temporal resolution (11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30). The training data were annotated by CLUST as ground truth (center of blood vessels) of fiducial features throughout the acquisition sequence.…”

“…Here, RNN efficiently handles temporal structure in sequences, while CNN handles spatial structures within single images. However, most of the approaches utilize convolutional and recurrent layers separately, followed by a fully connected layer, which may lead to a loss of spatial information …”

“…However, most of the approaches utilize convolutional and recurrent layers separately, followed by a fully connected layer, which may lead to a loss of spatial information. [18][19][20] The convolutional long short-term memory (CLSTM) RNN with forget gates and peephole connections is a modified version of vanilla RNN 21 and seems to be more suitable for the stated task of tumor tracking due to its track record for encoding long-range spatial information in sequential data and in circumventing the vanishing gradient problem. 22,23,24 CLSTM has convolutional structures in both the input-tostate and state-to-state transitions and has potential to model well the spatiotemporal relationships, while the traditional LSTM does not take spatial correlation into consideration.…”

Visual Tracking by Means of Deep Reinforcement Learning and an Expert Demonstrator

“…Our tracking agent maintains representations of both the policy π : S → A and the state value function v : S → R. This is done by using a DNN with parameters θ. In particular, we used a deep architecture that is similar to the one proposed by Gordon et al [10].…”

“…The proposed trackers are built on a deep regression network for tracking [13,10] and are trained inside an onpolicy Asynchronous Actor-Critic framework [32] that incorporates SL and expert demonstrations. A state-of-theart tracking algorithm [2] is run on a large-scale tracking dataset [18] to obtain the demonstrations.…”