Steered mixture-of-experts for light field video coding

Abstract

Steered Mixture-of-Experts (SMoE) is a novel framework for representing multidimensional image modalities. In this paper, we propose a coding methodology for SMoE models that is readily extendable to any dimensional SMoE model, thus representing any image modality of any dimension. We evaluate the coding performance of SMoE models of light field video, a 5D image modality, i.e. time, two angular, and two spatial dimensions. The coding consists of the exploiting the redundancy between the parameters of SMoE models, i.e. a set of multivariate Gaussian distributions. We compare the performance of three multi-view HEVC (MV-HEVC) configurations that differ in terms of random access. Each subaperture view from the light field video is interpreted as a single view in MV-HEVC. Experiments validate that excellent coding performance compared to MV-HEVC for low-to midrange bitrates in terms of PSNR and SSIM with bitrate savings up to 75%.

Sparse video representation using steered mixture-of-experts with global motion compensation

“…In the last couple of years the potential of the Steered Mixture-of-Experts (SMoE) framework for sparsely representing and coding images, 1-3 videos 4 and even higher dimensional imagery, i.e., light field images 5 and light field videos 6,7 has been investigated. As SMoE models give a description of the underlying data in the spatial domain instead using the frequency transform domain this compression approach drastically departs from conventional block-based coding techniques.…”

“…Such regions can be extended over the entire input space such that experts are responsible for thousands of pixels, 3 allowing for efficient representation of the image data. As the SMoE framework is easily scalable towards higher dimensional image modalities, this approach has been extended to video, 4 light field image 5 and light field video processing and coding, 6,7 respectively. They have shown that promising compression results can be achieved compared to state-of-the-art coding standards.…”