Rectifying rolling shutter video from hand-held devices

Forssén, Per-Erik; Ringaby, Erik

doi:10.1109/cvpr.2010.5540173

Cited by 97 publications

(98 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this article, we focus on a distortion model based on 3D camera rotation, since we have previously shown that it outperforms a combined rotation and translation model (Forssén and Ringaby, 2010). In this article, we extend the rotational model to use multiple knots across a frame.…”

Section: Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Video Rectification and Stabilisation for Cell-Phones

2011

Self Cite

View full text Add to dashboard Cite

This article presents a method for rectifying and stabilising video from cell-phones with rolling shutter (RS) cameras. Due to size constraints, cell-phone cameras have constant, or near constant focal length, making them an ideal application for calibrated projective geometry. In contrast to previous RS rectification attempts that model distortions in the image plane, we model the 3D rotation of the camera. We parameterise the camera rotation as a continuous curve, with knots distributed across a short frame interval. Curve parameters are found using non-linear least squares over inter-frame correspondences from a KLT tracker. By smoothing a sequence of reference rotations from the estimated curve, we can at a small extra cost, obtain a high-quality image stabilisation. Using synthetic RS sequences with associated ground-truth, we demonstrate that our rectification improves over two other methods. We also compare our video stabilisation with the methods in iMovie and Deshaker.

show abstract

Section: Contributionsmentioning

confidence: 99%

“…Even though this model is violated across an entire video clip, we have found it to work quite well if used on short frame intervals of 2-4 frames (Forssén and Ringaby, 2010). We represent the model as a sequence of rotation matrices, R(t) ∈ SO(3).…”

Section: Camera Motion Estimationmentioning

confidence: 99%

Efficient Video Rectification and Stabilisation for Cell-Phones

2011

Self Cite

View full text Add to dashboard Cite

show abstract

“…While the above approaches are rather 2D in nature, Forssen, Ringaby, Hedborg et al applied structure from motion algorithms to tackle the problem for static scenes: First, Forssen and Ringaby [6,20] had tracked features through cell phone video sequences and compensated cell phone rotation, which they identified as the dominant source of distortion for hand-held videos. In a later work, Hedborg et al [13] have shown a full bundle adjustment including motion effects as well.…”

Section: Previous Workmentioning

confidence: 99%

“…It has been shown recently that for hand-held smartphone cameras in static scenes, most of the rolling shutter effects can be compensated in the image (without 3D scene information), that is by compensating rotation [6,20,12,1]. However, in case a high resolution camera is mounted on a moderately fast driving capture vehicle, strong rolling shutter effects will be introduced by the motion of the camera, even if the camera orientation is stable (similarly at a smaller scale, for videobased reconstruction of objects using a cell phone).…”

Section: Introductionmentioning

confidence: 99%

Rolling Shutter Stereo

Saurer

Köser

Bouguet

et al. 2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

A huge fraction of cameras used nowadays is based on CMOS sensors with a rolling shutter that exposes the image line by line. For dynamic scenes/cameras this introduces undesired effects like stretch, shear and wobble. It has been shown earlier that rotational shake induced rolling shutter effects in hand-held cell phone capture can be compensated based on an estimate of the camera rotation. In contrast, we analyse the case of significant camera motion, e.g. where a bypassing streetlevel capture vehicle uses a rolling shutter camera in a 3D reconstruction framework. The introduced error is depth dependent and cannot be compensated based on camera motion/rotation alone, invalidating also rectification for stereo camera systems. On top, significant lens distortion as often present in wide angle cameras intertwines with rolling shutter effects as it changes the time at which a certain 3D point is seen. We show that naive 3D reconstructions (assuming global shutter) will deliver biased geometry already for very mild assumptions on vehicle speed and resolution. We then develop rolling shutter dense multiview stereo algorithms that solve for time of exposure and depth at the same time, even in the presence of lens distortion and perform an evaluation on ground truth laser scan models as well as on real street-level data.

show abstract

“…Much work has been done to recover object motion and shape from rolling-shutter images and video taken from fixed viewpoints [3] [4] [20]. Complementary work has been done to remove rolling shutter distortion from images and video created when the camera moves [18] [7] [9] [5] [13].…”

Section: Related Workmentioning

confidence: 99%