Non-causal Tracking by Deblatting

Abstract

0000−0001−9874−1349] , Jan Kotera 2[0000−0001−5528−5531] , FilipŠroubek 2[0000−0001−6835−4911] , and Jiří Matas 1[0000−0003−0863−4844]Abstract. Tracking by Deblatting 1 stands for solving an inverse problem of deblurring and image matting for tracking motion-blurred objects. We propose noncausal Tracking by Deblatting which estimates continuous, complete and accurate object trajectories. Energy minimization by dynamic programming is used to detect abrupt changes of motion, called bounces. High-order polynomials are fitted to segments, which are parts of the trajectory separated by bounces. The output is a continuous trajectory function which assigns location for every realvalued time stamp from zero to the number of frames. Additionally, we show that from the trajectory function precise physical calculations are possible, such as radius, gravity or sub-frame object velocity. Velocity estimation is compared to the high-speed camera measurements and radars. Results show high performance of the proposed method in terms of Trajectory-IoU, recall and velocity estimation. Recently, a method called Tracking by Deblatting 1 (TbD) has been introduced by Kotera et al.[5] to alleviate some of these restrictions. TbD performs significantly better than [15] and for a larger range of scenarios. The method solves two inverse problems of deblurring and image matting, and estimates object trajectories as piece-wise parabolic curves in each frame individually.In its core, TbD assumes causal processing of video frames, i.e. the trajectory reported in the current frame is estimated using only information from previous frames.

Tracking by Deblatting

“…This work is an extension of our earlier conference publications (Rozumnyi et al 2019) and (Kotera et al 2019). Some parts have been reported in Master Thesis by (Rozumnyi 2019).…”

Progressive-X+: Clustering in the Consensus Space

“…Input image I and background B are assumed to be known. The unknowns in (2) are estimated either by alternating energy minimization with additional priors [30,48,46,31,29], or more recently by learning from synthetic data [50,49]. The formation model in (2) encodes the trajectory by the blur kernel.…”

Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

“…where the motion blur is modeled by the convolution of the sharp object appearance F and its trajectory, defined by the blur kernel H. Several follow-up methods [13,14,25,26,28,29,39] were proposed to solve for (F, M, H) given the input image I and background B. They approximate the solution in a least-squares sense by energy minimization with suitable regularizers summarized by function reg(•):…”