Joint Video Stitching and Stabilization From Moving Cameras

Abstract

In this paper, we extend image stitching to video stitching for videos that are captured for the same scene simultaneously by multiple moving cameras. In practice, videos captured under this circumstance often appear shaky. Directly applying image stitching methods for shaking videos often suffers from strong spatial and temporal artifacts. To solve this problem, we propose a unified framework in which video stitching and stabilization are performed jointly. Specifically, our system takes several overlapping videos as inputs. We estimate both inter motions (between different videos) and intra motions (between neighboring frames within a video). Then, we solve an optimal virtual 2D camera path from all original paths. An enlarged field of view along the virtual path is finally obtained by a space-temporal optimization that takes both inter and intra motions into consideration. Two important components of this optimization are that: 1) a grid-based tracking method is designed for an improved robustness, which produces features that are distributed evenly within and across multiple views and 2) a mesh-based motion model is adopted for the handling of the scene parallax. Some experimental results are provided to demonstrate the effectiveness of our approach on various consumer-level videos and a Plugin, named "Video Stitcher" is developed at Adobe After Effects CC2015 to show the processed videos.

Multi-Frame Based Homography Estimation for Video Stitching in Static Camera Environments

“…Experiments for performance evaluation consist of three parts. The first experiment compares the alignment distortion and stitching score [17] of the aligned frames after applying the proposed and the per-frame method to daytime video sequences. The second experiment also compares the alignment distortion and the stitching score for noisy video sequences to test the robustness of the estimated homography against noise.…”

“…In dynamic camera environments, the homography between two cameras may change with time. Therefore, the homography between the first input frames of a pair of video sequences to be stitched is extracted, the motion for each video sequence is calculated using the optical flow algorithm, and the new frame-specific homography is updated using the initial homography and the frame-specific motion [16,17].…”

Content-Preserving Image Stitching With Piecewise Rectangular Boundary Constraints

“…Lin et al [22] firstly proposed a robust framework to stitch videos from moving hand-held cameras, which incorporates stabilization and stitching into a unified framework. Guo et al [23] and Nei et al [24] further improved the performance of a joint video stabilization and stitching framework. Their main contributions include: estimation of inter-motions between cameras and intra-motions in a video, and common background identification for multiple input videos.…”

Distortion-free wide-angle portraits on camera phones

“…et Wolf et al 2007], texture deformation [Gal et al 2006], stereoscopic editing [Chang et al 2011;Du et al 2013], and video stabilization [Guo et al 2016;Liu et al 2009Liu et al , 2011Zhang et al 2016]. Different from existing methods that restore geometries of generic objects, our work specifically addresses portrait photos.…”