A universal image coding approach using sparse steered Mixture-of-Experts regression

Abstract

To refer to or to cite this work, please use the citation to the published version: Verhack, R., Sikora, T., Lange, L., Van Wallendael, G., and Lambert, P. (2016 Ghent University -iMinds -Data Science Lab, Ghent, Belgium † Technische Universität Berlin -Communication Systems Lab, Berlin, Germany ABSTRACT Our challenge is the design of a "universal" bit-efficient image compression approach. The prime goal is to allow reconstruction of images with high quality. In addition, we attempt to design the coder and decoder "universal", such that MPEG-7-like low-and mid-level descriptors are an integral part of the coded representation. To this end, we introduce a sparse Mixture-of-Experts regression approach for coding images in the pixel domain. The underlying stochastic process of the pixel amplitudes are modelled as a 3-dimensional and multi-modal Mixture-of-Gaussians with K modes. This closed form continuous analytical model is estimated using the Expectation-Maximization algorithm and describes segments of pixels by local 3-D Gaussian steering kernels with global support. As such, each component in the mixture of experts steers along the direction of highest correlation. The conditional density then serves as the regression function. Experiments show that a considerable compression gain is achievable compared to JPEG for low bitrates for a large class of images, while forming attractive low-level descriptors for the image, such as the local segmentation boundaries, direction of intensity flow and the distribution of these parameters over the image.

Color prediction in image coding using Steered Mixture-of-Experts

“…Consequently, we arrive at a closed form continuous analytical model. Previous work has illustrated this idea for image and video luma coding, where the joint probability density function was modeled as respectively 3-D and 4-D Gaussian Mixture Models (GMMs) [9] [10].…”

“…entropy coding) is performed to the values in this work. More elaborate methods exist using difference coded components, eigen decompositions, and arithmetic coding [9].…”

“…In this work we take a completely different approach, which was largely motivated by the recently introduced Steered Mixture-of-Experts Regression (SMoE) methodology [9][10], combined with ideas from image colorization [11]. We present a color modeling scheme using Mixture-of-Experts to model a non-linear predictor F (x, Y ) → (C b , C r ) over the whole image, where Y is the luminance, x is the 2-dimensional pixel location, and C b and C r are the chrominance values.…”

Steered mixture-of-experts for light field coding, depth estimation, and processing

“…Verhack et al and Lange et al proposed the Steered Mixture-of-Experts (SMoE) framework for images and video [1] [2]. This work proposes the SMoE framework for light fields (LFs).…”

“…The expected value of the conditional density then serves as the regression function. Previous work has illustrated this idea for image and video luma coding, where the joint probability density function was modeled as respectively 3-D and 4-D Gaussian Mixture Models [1][2], and is here extended to color LFs.…”