Light-field image compression based on variational disparity estimation and motion-compensated wavelet decomposition

Abstract

This paper presents a compression framework for light-field images. The main idea of our approach is exploiting the similarity across sub-aperture images extracted from light-field data to improve encoding performance. For this purpose we propose a variational optimisation approach to estimate the disparity map from light-field images and then apply it to a motion-compensated wavelet lifting scheme. Making use of JPEG2000 for coding all high-/low-pass sub-band views as well as disparity map, our approach can therefore support both lossless and lossy compression. The coding framework is tested with both synthetic and real-world light-field dataset. The experimental results demonstrate that our approach outperforms JPEG-LS and the direct application of JPEG2000 in both lossless and lossy compression scenarios.

Rate-Distortion Optimized Graph Coarsening and Partitioning for Light Field Coding

“…While the above techniques significantly rely on blockbased prediction mechanisms of standardized solution, methods departing from classical Inter-coding tools have also been proposed, as in [32] where the author exploits interview correlation by using homography and 2D warping to predict views, or as in [31] where the authors use a linear approximation computed with Matching Pursuit for disparity based view prediction. View synthesis techniques have also been considered in [31], [33], [34] for reconstructing the entire light field from a sparse set of views, using either depth image-based rendering techniques [34], [35], convolutional neural networks [33], or linear approximation computed with Matching Pursuit for disparity based view prediction [31]. The synthesized set of views are then used as predictors of the original light field views.…”

Local Low Rank Approximation With a Parametric Disparity Model for Light Field Compression

“…While the above techniques significantly rely on blockbased prediction mechanisms of standardized solution, other compression schemes instead use efficient view synthesis techniques for first reconstructing the entire light field from a very sparse set of views [28], [27], [29]. In [28], the authors use a convolutional neural network to predict all views from the four corner ones, while the authors in [29] and [30] rather follow a depth image-based rendering approach in which depth is first estimated and used to warp reference views to predict the others. In [29], the warping is done per segmented region of a reference view (or a set of reference views).…”

“…The resulting multiple references are then used to predict the others using a sparse predictor. The authors in [30] apply a disparity compensated wavelet coding technique. Disparityguided sparse coding methods with learned dictionaries are instead considered in [31], while the authors in [27] use a linear approximation computed with Matching Pursuit for disparity based view prediction.…”

“…This is explained by the fact that the affine model better aligns the super-pixels of each super-ray, which reduces the low-rank approximation error. At higher bitrates (> 0.1 bpp), corresponding to higher rank values (10,15,30,60) the lesser alignment of super-pixels is compensated by the additional information allowed to be sent.…”

Coding of Light Fields Using Disparity-Based Sparse Prediction

“…The sparse set and the disparities are encoded with HEVC and the rest of the plenoptic image is synthesized using disparity based reconstruction, interpolation, and inpainting. In [30] a motion compensated wavelet lifting scheme is used to encode the sub-aperture images. Similar to our encoding strategy, the disparity map is first estimated and subsequently encoded in the codestream.…”