Learning to Extract a Video Sequence from a Single Motion-Blurred Image

Jin, Meiguang; Meishvili, Givi; Favaro, Paolo

doi:10.1109/cvpr.2018.00663

Cited by 121 publications

(233 citation statements)

References 18 publications

Supporting

Mentioning

231

Contrasting

Unclassified

Order By: Relevance

“…Autocorrelation techniques require a relatively large neighborhood to estimate blur parameters and such methods are not suitable for small moving objects. More recently, deep learning has been applied to motion deblurring of videos [37,31] and to the generation of intermediate short-exposure frames [14]. The proposed convolutional neural networks are trained only on small blurs; blur parameters are not available as they are not directly estimated.…”

Section: Related Workmentioning

confidence: 99%

Intra-Frame Object Tracking by Deblatting

Kotera¹,

Rozumnyi²,

Šroubek³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects elapse non-negligible distance during exposure time of a single frame and therefore their position in the frame is not well defined. They appear as semitransparent streaks due to the motion blur and cannot be reliably tracked by standard trackers.We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. The trajectory is then estimated by fitting a piecewise quadratic curve, which models physically justifiable trajectories. As a result, tracked objects are precisely localized with higher temporal resolution than by conventional trackers.The proposed TbD tracker was evaluated on a newly created dataset of videos with ground truth obtained by a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms baseline both in recall and trajectory accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

Intra-Frame Object Tracking by Deblatting

Kotera¹,

Rozumnyi²,

Šroubek³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…Gong et al [7] learned optical flow field from a single blurry image directly through a fullyconvolutional deep neural network and recovered the clean image from the learned optical flow. Jin et al [15] extracted a video sequence from a single motion-blurred image by introducing loss functions invariant to the temporal order. Li et al [22] used a learned image prior to distinguish whether an image is sharp or not and embedded the learned prior into the MAP framework.…”

Section: Related Workmentioning

confidence: 99%

Single Image Deblurring and Camera Motion Estimation With Depth Map

Pan

Dai

Liu

2019

2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Camera shake during exposure is a major problem in hand-held photography, as it causes image blur that destroys details in the captured images. In the real world, such blur is mainly caused by both the camera motion and the complex scene structure. While considerable existing approaches have been proposed based on various assumptions regarding the scene structure or the camera motion, few existing methods could handle the real 6 DoF camera motion. In this paper, we propose to jointly estimate the 6 DoF camera motion and remove the non-uniform blur caused by camera motion by exploiting their underlying geometric relationships, with a single blurry image and its depth map (either direct depth measurements, or a learned depth map) as input. We formulate our joint deblurring and 6 DoF camera motion estimation as an energy minimization problem which is solved in an alternative manner. Our model enables the recovery of the 6 DoF camera motion and the latent clean image, which could also achieve the goal of generating a sharp sequence from a single blurry image. Experiments on challenging real-world and synthetic datasets demonstrate that image blur from camera shake can be well addressed within our proposed framework.

show abstract

“…One application we explore is the formation of videos from dramatically motion-blurred images, created by temporally aggregating photons from a scene over an extended period of time. Two recent studies present the deterministic recovery of a video sequence from a single motion-blurred image [18,30]. We propose a general deprojection framework for dimensions including, but not limited to time.…”

Section: Inverting a Motion-blurred Image To Videomentioning

confidence: 99%

“…The digits can occlude one another, and bounce off the edges of the frames. Given a dataset of 64 × 64 × 10-sized video subclips, we generate each projection x by averaging the frames in time, similar to other studies that generate motion-blurred images at a large scale [18,21,27,28]. Despite the simple appearance and dynamics of this dataset, synthesizing digit appearances and capturing the plausible directions of each trajectory is challenging.…”

Section: Temporal Deprojections With Moving Mnistmentioning

confidence: 99%

Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions

Balakrishnan

Dalca

Zhao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

We introduce visual deprojection: the task of recovering an image or video that has been collapsed along a dimension. Projections arise in various contexts, such as longexposure photography, where a dynamic scene is collapsed in time to produce a motion-blurred image, and corner cameras, where reflected light from a scene is collapsed along a spatial dimension because of an edge occluder to yield a 1D video. Deprojection is ill-posed-often there are many plausible solutions for a given input. We first propose a probabilistic model capturing the ambiguity of the task. We then present a variational inference strategy using convolutional neural networks as functional approximators. Sampling from the inference network at test time yields plausible candidates from the distribution of original signals that are consistent with a given input projection. We evaluate the method on several datasets for both spatial and temporal deprojection tasks. We first demonstrate the method can recover human gait videos and face images from spatial projections, and then show that it can recover videos of moving digits from dramatically motion-blurred images obtained via temporal projection.

show abstract

Learning to Extract a Video Sequence from a Single Motion-Blurred Image

Cited by 121 publications

References 18 publications

Intra-Frame Object Tracking by Deblatting

Intra-Frame Object Tracking by Deblatting

Single Image Deblurring and Camera Motion Estimation With Depth Map

Visual Deprojection: Probabilistic Recovery of Collapsed Dimensions

Contact Info

Product

Resources

About