MPEG Immersive Video Coding Standard

Boyce, Jill M.; Doré, Renaud; Dziembowski, Adrian; Fleureau, Julien; Jung, Joël; Kroon, Bart; Salahieh, Basel; Vadakital, Vinod Kumar Malamal; Yu, Lu

doi:10.1109/jproc.2021.3062590

Cited by 71 publications

(31 citation statements)

References 18 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While it is possible to discard several views before encoding the quality of the reconstructed scene may be reduced. The other solution is denoted as MPEG Immersive Video (MIV) [4]. While its reference software implementation uses HEVC, it is agnostic to the video codec.…”

Section: Introductionmentioning

confidence: 99%

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Garus

Henry

Jung

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

Immersive video often refers to multiple views with texture and scene geometry information, from which different viewports can be synthesized on the client side. To design efficient immersive video coding solutions, it is desirable to minimize bitrate, pixel rate and complexity. We investigate whether the classical approach of sending the geometry of a scene as depth maps is appropriate to serve this purpose. Previous work shows that bypassing depth transmission entirely and estimating depth at the client side improves the synthesis performance while saving bitrate and pixel rate. In order to understand if the encoder side depth maps contain information that is beneficial to be transmitted, we first explore a hybrid approach which enables partial depth map transmission using a block-based RD-based decision in the depth coding process. This approach reveals that partial depth map transmission may improve the rendering performance but does not present a good compromise in terms of compression efficiency. This led us to address the remaining drawbacks of decoder side depth estimation: complexity and depth map inaccuracy. We propose a novel system that takes advantage of high quality depth maps at the server side by encoding them into lightweight features that support the depth estimator at the client side. These features allow reducing the amount of data that has to be handled during decoder side depth estimation by 88%, which significantly speeds up the cost computation and the energy minimization of the depth estimator. Furthermore, -46.0% and -37.9% average synthesis BD-Rate gains are achieved compared to the classical approach with depth maps estimated at the encoder.

show abstract

Section: Introductionmentioning

confidence: 99%

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Garus

Henry

Jung

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, all tested configurations were compared against the DSDE anchor in CTC called "G17 anchor." The G17 anchor was generated using the GA profile of the MIV standard [9] and the test model for MIV 8 (TMIV 8) [24]. When using the GA profile, the MIV encoder selects a subset of input views to decrease the total bitrate and pixel rate [25] of the transmitted video.…”

Section: Experiments 41 | Methodologymentioning

confidence: 99%

“…As immersive video applications gain widespread attention in the video processing community, there has been a noticeable increase in standardization efforts in this field [8]. One of the currently prepared standards, that is, MPEG Immersive video (MIV) [9], provides state-of-the-art methods for efficient encoding of multiview sequences. Although its main profile focuses on removing interview redundancy and packing of remaining data, the "geometry absent" (MIV GA) profile follows the idea of estimating the geometry of the scene at the decoder side [10].…”

Section: Introductionmentioning

confidence: 99%

Recursive block splitting in feature‐driven decoder‐side depth estimation

et al. 2022

Self Cite

View full text Add to dashboard Cite

This paper presents a study on the use of encoder-derived features in decoderside depth estimation. The scheme of multiview video encoding does not require the transmission of depth maps (which carry the geometry of a threedimensional scene) as only a set of input views and their parameters are compressed and packed into the bitstream, with a set of features that could make it easier to estimate geometry in the decoder. The paper proposes novel recursive block splitting for the feature extraction process and evaluates different scenarios of feature-driven decoder-side depth estimation, performed by assessing their influence on the bitrate of metadata, quality of the reconstructed video, and time of depth estimation. As efficient encoding of multiview sequences became one of the main scopes of the video encoding community, the experimental results are based on the "geometry absent" profile from the incoming MPEG Immersive video standard. The results show that the quality of synthesized views using the proposed recursive block splitting outperforms that of the state-of-the-art approach.

show abstract

“…Therefore, increasing the coding efficiency of depth maps of 6DoF videos is crucial for providing an immersive visual experience. Some studies proposed the reduction of the depth information errors that may occur in the representation of depth with a finite number of bits intended for better coding efficiency [17], [18].…”

Section: Introductionmentioning

confidence: 99%

Nonlinear Depth Quantization Using Piecewise Linear Scaling for Immersive Video Coding

Park

Lim

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Moving Picture Experts Group (MPEG) is developing a standard for immersive video coding called MPEG Immersive Video (MIV) and is releasing a reference software called Test Model for Immersive Video (TMIV) in the standardization process. The TMIV efficiently compresses an immersive video comprising a set of texture and depth views acquired using multiple cameras within a limited 3D viewing space. Moreover, it affords a rendered view of an arbitrary view position and orientation with six degrees of freedom (6DoF). However, the existing depth quantization applied to depth atlas in TMIV is insufficient since the reconstructed depth is crucial for achieving the required quality of a rendered viewport. To address this issue, we propose a nonlinear depth quantization method that allocates more codewords to a depth subrange with a higher occurrence of depth values located at edge regions, which are important in terms of the rendered view quality. We implement the proposed nonlinear quantization based on piecewise linear scaling considering the computational complexity and bitstream overhead. The experimental results show that the proposed method yields PSNR-based Bjøntegaard delta rate gains of 5.2% and 4.9% in the end-toend performance for High-and Low-bitrate (BR) ranges, respectively. Moreover, subjective quality improvement is mainly observed at the object boundaries of the rendered viewport. The proposed nonlinear quantization method has been adopted into the TMIV as a candidate standard technology for the next MIV edition.

show abstract

MPEG Immersive Video Coding Standard

Cited by 71 publications

References 18 publications

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Immersive Video Coding: Should Geometry Information Be Transmitted as Depth Maps?

Recursive block splitting in feature‐driven decoder‐side depth estimation

Nonlinear Depth Quantization Using Piecewise Linear Scaling for Immersive Video Coding

Contact Info

Product

Resources

About