Intra-Frame Object Tracking by Deblatting

Abstract

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects elapse non-negligible distance during exposure time of a single frame and therefore their position in the frame is not well defined. They appear as semitransparent streaks due to the motion blur and cannot be reliably tracked by standard trackers.We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. The trajectory is then estimated by fitting a piecewise quadratic curve, which models physically justifiable trajectories. As a result, tracked objects are precisely localized with higher temporal resolution than by conventional trackers.The proposed TbD tracker was evaluated on a newly created dataset of videos with ground truth obtained by a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms baseline both in recall and trajectory accuracy.

Non-causal Tracking by Deblatting

“…Experiments are done on the TbD dataset [5] with the ground truth trajectories from a high-speed camera. We use Trajectory Intersection over Union (TIoU) proposed by Table 1.…”

“…TbD [5] is the only method that uses motion blur and deblurring to improve tracking results and performs parametric fit to estimate intra-frame trajectories. The TbD dataset presented therein is another dataset with FMOs which contains ground truth trajectories and can be used for evaluating trajectory accuracy.…”

“…A brief overview of TbD follows. The acquisition model with fast moving objects proposed in [15,5] is defined as…”

“…Frame t Speed [r/ ] GT TbD TbD-NC Tracking by Deblatting [5] Non-causal trajectory estimation Speed estimation Fig. 1.…”

“…1. Trajectory reconstruction using the proposed non-causal Tracking by Deblatting (middle) compared to the causal TbD [5] (left). Color denotes the trajectory accuracy, from red (complete failure) to green (high accuracy).…”