3D Geometry-Based Global Motion Compensation For VVC

Abstract

Since most 2D videos are initially captured in 3D environments, developing 3D motion models for video coding is beneficial. This paper introduces a method for extracting 3D geometry data from 2D videos and synthesizing 3D-based virtual Reference Pictures (RPs). These novel RPs are offered to the Versatile Video Coding (VVC) encoder for use in motion compensation. The proposed method generates 3D geometry data in the form of 3D meshes at the encoder and transmits it to the decoder as an overhead bitstream. However, this overhead could eat up the whole coding gain or even make it worse than the anchor VVC. This paper solves this problem by employing 3D mesh processing techniques, e.g., mesh denoising, mesh decimation, and mesh compression. Simulation results show that the proposed method outperforms VVC up to ~3.8%.

Video Global Motion Compensation Based on Affine Inverse Transform Model

“…Moving object detection under a dynamic background [ 16 , 17 ] mainly includes optical flow method [ 18 , 19 ] and global motion compensation method [ 20 , 21 ]. The global motion compensation method first estimates the global motion and then analyzes and calculates the motion parameters for motion compensation.…”