A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image

Abstract

The multi-view video is a collection of multiple videos, capturing the same scene at different viewpoints. Since it contains more affluent information than a single video, it can be applied to various applications, such as 3DTV, free viewpoint TV, surveillance, sports matches, and so on. However, the data size of the multi-view video linearly increases as the number of cameras, therefore it is necessary to develop an effective framework to represent, process, and transmit those huge amounts of data. In recent, multi-view video coding is getting lots of attention as efficient video coding technologies are being developed. Although most of multi-view video coding algorithms are based on the state-of-the-art H.264/AVC video coding technology, they do not utilize rich 3-D information. In this paper, we propose a new framework using the concept of layered depth image (LDI), one of the efficient image-based rendering techniques, to efficiently represent and process multi-view video data. We describe how to represent natural multi-view video based on the LDI approach and the overall framework to process those converted data.

Incremental-LDI for multi-view coding

“…Given a set of viewpoints and one depth map per view, the classical algorithm for LDI construction [6,9] proceeds in three steps, summarized in figure 1. First, an arbitrary viewpoint is chosen as the reference viewpoint (it is usually chosen among input viewpoints).…”

“…The Layer Depth Image (LDI) representation [5,6] is one of these IBR approaches. In this representation, pixels are no more composed by a single color and a single depth value, but can contain several colors and associated depth values.…”

“…Various approaches to LDI compression have been proposed [6,7,8], based on classical LDI's layers constructions [6,9]. The problem is that layers generated are still correlated, and some pixels are redundant between layers.…”

Object-based Layered Depth Images for improved virtual view synthesis in rate-constrained context

“…The Layer Depth Image (LDI) representation [5,6] is one of these IBR approaches. It extends the 2D+Z representation, but instead of representing the scene with an array of depth pixels (pixel color with associated depth values), each position in the array may store several depth pixels, organised into layers.…”

“…Various approaches to LDI construction have been proposed [6,7,8]. All of them organize layers by visibility.…”