Deep-learning-based macro-pixel synthesis and lossless coding of light field images

Abstract

Deep-learning-based macro-pixel synthesis and lossless coding of light field imagesionut schiopu and adrian munteanu This paper proposes a novel approach for lossless coding of light field (LF) images based on a macro-pixel (MP) synthesis technique which synthesizes the entire LF image in one step. The reference views used in the synthesis process are selected based on four different view configurations and define the reference LF image. This image is stored as an array of reference MPs which collect one pixel from each reference view, being losslessly encoded as a base layer. A first contribution focuses on a novel network design for view synthesis which synthesizes the entire LF image as an array of synthesized MPs. A second contribution proposes a network model for coding which computes the MP prediction used for lossless encoding of the remaining views as an enhancement layer. Synthesis results show an average distortion of 29.82 dB based on four reference views and up to 36.19 dB based on 25 reference views. Compression results show an average improvement of 29.9 over the traditional lossless image codecs and 9.1 over the state-of-the-art.

Epipolar Plane Image-Based Lossless and Near-Lossless Light Field Compression

“…This method achieves better performance than CALIC, however, the network has around a million parameters which makes the prediction process computational intensive for a regular PC. In [7], the authors achieve lossless LF compression in two stages. In the first stage, a network MPS-CNN synthesize the full LF image from a sub-sampled LF encoded using REP-CNN [23].…”

“…11 reveals that, as the neighboring region gets smoother, the estimated slopê d = ∇ y /∇ x becomes unreliable. To prevent this from causing a poor intensity prediction, this problem is treated by assigning more weight to the average of the neighboring intensities in (7). However, there is still some contribution of the poor prediction in the final estimate.…”

“…Some applications can not tolerate distortion artifacts introduced by compression algorithms, such as medical diagnosis, where the distortion may remove or obscure very significant and life-saving information, while possibly introducing misleading artifacts into the image [5]. The lossless LF compression schemes in the literature offer decent compression ratios at the cost of hours of encoding time [6] and/or complex computation [7]. For this reason, we have recently proposed an efficient lossless LF compression scheme, EPIC [8], which extends CALIC to exploit the LF structure for enhanced compression performance.…”

Attention Networks for the Quality Enhancement of Light Field Images

“…In our prior work, research efforts were invested to provide innovative solutions for LF coding based on efficient Deep-Learning (DL)-based prediction methods [ 20 , 29 , 30 , 31 , 32 ] and CNN-based filtering methods for quality enhancement [ 33 , 34 ]. In [ 29 ], we introduced a lossless codec for LF images based on context modeling of SAI images.…”

“…In [ 31 ], we proposed to employ a DL-based method to synthesize an entire LF image based on different configurations of reference SAIs and then to employ an MP-wise prediction method to losslessly encode the remaining views. In [ 32 ], we proposed a residual-error prediction method based on deep learning and a context-tree based bit-plane codec, where the experimental evaluation was carried out on photographic images, LF images, and video sequences.…”