Object-Based Coding for Plenoptic Videos

Abstract

Abstract-A new object-based coding system for a class of dynamic image-based representations called plenoptic videos (PVs) is proposed. PVs are simplified dynamic light fields, where the videos are taken at regularly spaced locations along line segments instead of a 2-D plane. In the proposed object-based approach, objects at different depth values are segmented to improve the rendering quality. By encoding PVs at the object level, desirable functionalities such as scalability of contents, error resilience, and interactivity with an individual image-based rendering (IBR) object can be achieved. Besides supporting the coding of texture and binary shape maps for IBR objects with arbitrary shapes, the proposed system also supports the coding of grayscale alpha maps as well as depth maps (geometry information) to respectively facilitate the matting and rendering of the IBR objects. Both temporal and spatial redundancies among the streams in the PV are exploited to improve the coding performance, while avoiding excessive complexity in selective decoding of PVs to support fast rendering speed. Advanced spatial/temporal prediction methods such as global disparity-compensated prediction, as well as direct prediction and its extensions are developed. The bit allocation and rate control scheme employing a new convex optimization-based approach are also introduced. Experimental results show that considerable improvements in coding performance are obtained for both synthetic and real scenes, while supporting the stated object-based functionalities.Index Terms-IBR object coding, image-based rending, MPEG-4, object-based coding, plenoptic videos.

User-aware Video Coding Based on Semantic Video Understanding and Enhancing

“…Many researches (Cheng et al, 2008;Bertini et al, 2006;Ng et al, 2010) investigate methodologies for analyzing the semantic meaning of the video. Within the MPEG-7 (ISO/IEC 15938) Multimedia Description Schemes specification, "event" is used in the Creation and Production description tools to describe the agents and …”

“…To support the interactivity of video coding, many researches (Naman & Taubman, 2007;Ng et al, 2010;Wang et al, 2005;Tran et al, 2004) propsed content-based scalable coding schemes, most of which applied the concept of VOPs of MPEG-4. Fig.…”

The Design and Construction of a Movable Image-Based Rendering System and Its Application to Multiview Conferencing

“…In particular, the IBR objects can be encoded individually by an MPEG-4 like object-based coding scheme [32,33] which also includes additional information such as depth maps and alpha maps to facilitate rendering. Moreover, the userdefined IBR objects can be flexibly reconstructed at the decoder for rendering and other processing.…”

“…For the captured plenoptic videos, the multiple videos can be compressed offline using the object-based coder we have proposed in [32,33]. It is based on the MPEG-4 coder and the picture frame structure is shown in Fig.…”

“…By extending the concept of GOVOPF, group of frame fields can be collected to form group of frame fields (GOFF). Interested readers are referred to [32,33] for more details.…”