Low-complexity multiview video coding

Abstract

Abstract3D video is a technology that has seen a tremendous attention in the recent years. Multiview Video Coding (MVC) is an extension of the popular H.264 video coding standard and is commonly used to compress 3D videos. It offers an improvement of 20% to 50% in compression efficiency over simulcast encoding of multiview videos using the conventional H.264 video coding standard. However, there are two important problems associated with it: (i) its superior compression performance comes at the cost of significantly higher computational complexity which hampers the real-world realization of MVC encoder in applications such as 3D live broadcasting and interactive Free Viewpoint Television (FTV), and (ii) compressed 3D videos can suffer from packet loss during transmission, which can degrade the viewing quality of the 3D video at the decoder. This thesis aims to solve these problems by presenting techniques to reduce the computational complexity of the MVC encoder and by proposing a consistent error concealment technique for frame losses in 3D video transmission.The thesis first analyses the complexity of the MVC encoder. It then proposes two novel techniques to reduce the complexity of motion and disparity estimation. The first method achieves complexity reduction in the disparity estimation process by exploiting the relationship between temporal levels, type of macroblocks and search ranges while the second method achieves it by exploiting the geometrical relationship between motion and disparity vectors in stereo frames. These two methods are then combined with other state-of-the-art methods in a unique framework where gains add up. Experimental results show that the proposed low-complexity framework can reduce the encoding time of the standard MVC encoder by over 93% while maintaining similar compression efficiency performance.The addition of new View Synthesis Prediction (VSP) modes to the MVC encoding framework improve the compression efficiency of MVC. However, testing additional modes comes at the cost of increased encoding complexity. In order to reduce the encoding complexity, the thesis, next, proposes a bayesian early mode decision technique for a VSP enhanced MVC coder. It exploits the statistical similarities between the RD costs of the VSP SKIP mode in neighbouring views to terminate the mode decision process early. Results indicate that the proposed technique can reduce the encoding time of the enhanced MVC coder by over 33% at similar compression efficiency levels.Finally, compressed 3D videos are usually required to be broadcasted to a large number of users where transmission errors can lead to frame losses which can degrade the video quality at the decoder. A simple reconstruction of the lost frames can lead to inconsistent reconstruction of the 3D scene which may negatively affect the viewing experience of a user. In order to solve this problem, the thesis proposes, at the end, a consistency model for recovering frames lost during transmission. The proposed consistency model is used to evaluate...

Fast Encoder Design for Multi-view Video

“…For sequences containing complicated motion and less disparity, such as Race1 and Ballroom, the time saving is lower, but still significant, at about 70.32-82.87%. To the best of our knowledge, the state-of-the-art results are those reported in [22], which combines state-of-the-art methods ( [19], [20], and [21]) with its PDV-DE technique. Compared with the methods in [19], [20], and [21], PDV-DE increases the time saving by only 1.5%.…”

“…To the best of our knowledge, the state-of-the-art results are those reported in [22], which combines state-of-the-art methods ( [19], [20], and [21]) with its PDV-DE technique. Compared with the methods in [19], [20], and [21], PDV-DE increases the time saving by only 1.5%. To evaluate the performance of each algorithm alone, we further compare the MVC methods in [19] and [20] with our overall MVC encoding technique proposed in Section 2.…”

“…The approach in [20] increases the overall speed of MVC by reducing the size of the search range for both ME and DE in regions with homogeneous motion. Inter-view SKIP mode correlation has been exploited by Shen et al [21]. Reference [22] reports a low-complexity DE technique that chooses the previous disparity vector (PDV) instead of the median prediction vector as the search center for DE.…”

“…Reference [22] reports a low-complexity DE technique that chooses the previous disparity vector (PDV) instead of the median prediction vector as the search center for DE. By combining the MVC methods in [19][20][21] with PDV-DE, a novel complexity reduction technique can be derived. The correlation of rate distortion (RD) costs among inter-views in SKIP mode are studied and used to reduce the computational complexity of MVC in [23].…”