Full Parallax 3D Video Content Compression

Abstract

Motion parallax is a key cue in the perception of the depth that current 3D stereoscopy and auto-stereoscopy technologies are not able to reproduce. Integral imaging and Super Multi-View video (SMV) are 3D technologies that allow creating a light-field representation of a scene with a smooth full motion parallax (i.e., in horizontal and vertical directions). However the large amount of data required is challenging and implies a need for new efficient coding technologies. This chapter first describes integral imaging and SMV content, acquisition, and display. Then it provides an overview of state-of-the-art methods for full parallax 3D content compression. Finally, several coding schemes are compared and a coding structure that exploits inter-view correlations in both horizontal and vertical directions is proposed. The new structure provides a rate reduction (for the same quality) up to 29.1 % when compared to a basic anchor structure. Neighboring Block Disparity Vector (NBDV) and Inter-View Motion Prediction (IVMP) coding tools are further improved to efficiently exploit coding structures in two dimensions, with rate reduction up to 4.2 % with respect to the reference 3D-HEVC encoder.

Light Field Compression With Disparity-Guided Sparse Coding Based on Structural Key Views

“…Kundu et al [25] exploited inter-view correlation by applying homography and 2D warping for view prediction. Relevant research in multi-view video coding [26], full parallax 3D video content compression [27], and holoscopic video coding [22] can be efficiently applied on LF data. Sakamoto et al [28] proposed to reconstruct the LF from multi-focus images, which contain mostly low frequency components and therefore easier to compress.…”

Light Field Coding With Field-of-View Scalability and Exemplar-Based Interlayer Prediction

“…Some other coding schemes proposed to extract the viewpoint images (VIs) from the LF content to be represented as multiview content [21]- [24], or as a Pseudo Video Sequence (PVS) [25]- [28]. In these coding approaches, each VI is constructed by simply extracting one pixel with the same relative position from each MI.…”

“…In these coding approaches, each VI is constructed by simply extracting one pixel with the same relative position from each MI. The VI-based multiview content is then encoded using a 3D video coding solution, for instance, the Multiview Video Coding (MVC) [29] (in [21]- [23]) and the multiview extension of HEVC (MV-HEVC) [30] (in [24]). An advantage of using a standard 3D video coding solution is that scalability and backward compatibility are straightforwardly supported.…”