Segmentation-Based Seam Cutting for High-Resolution 360-Degree Video Stitching

Abstract

We present a novel segmentation-based seam cutting algorithm to generate visually plausible high-resolution 360-degree video efficiently. While the demand for an efficient video stitching algorithm for generating immersive videos has increased, it has received limited attention in the literature. Furthermore, stitched videos often suffer from distorted objects, temporal inconsistency, and time constraints. Thus, in this paper, we propose an efficient seam finding algorithm that preserves objects from distortion, minimizes temporal inconsistency and reduces processing time. One of the fundamental steps in image and video stitching is the estimation of seam boundary. To do this, the proposed algorithm leverages a convolutional neural networks-based instance segmentation algorithm that provides more accurate object regions. It computes energy surfaces considering the regions and then estimates seam boundary by discovering a minimal energy path with minimal computations. We validate the proposed algorithm using challenging high-resolution 360-degree virtual reality sequences. The experimental results verify that the proposed algorithm can produce seam boundaries that avoid objects with better temporary consistency. The proposed algorithm reduces the number of pixels passed through objects by approximately 30% on average compared to the existing algorithms. The qualitative comparisons furthermore demonstrate that the proposed algorithm consistently produces more perceptually pleasing results.