Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Abstract

Because of the capacity of capturing both the spatial and angular information of the light rays simultaneously, light field images (LFIs) contain richer scene information compared with conventional images, but at the cost of huge volume. This paper proposes a novel LFI sparse compression framework driven by convolutional neural network (CNN). The epipolar plane image (EPI) super-resolution is for compensating the information loss caused by sparse sampling and the decoder-side sub-aperture images (SAIs) quality enhancement is for compensating the information loss caused by lossy compression. Specifically, we choose those SAIs both in odd rows and odd columns as our key SAIs and compress them using standard video encoder. For those non-key SAIs, we predict them using decompressed key SAIs by taking advantage of the special structure of EPI. The low-resolution EPIs generated from the sparse SAIs are super-resolved by a CNN and the outputs, high-resolution EPIs, are used to rebuild the dense SAIs. Moreover, in order to improve the quality of the predicted SAI, we add decoder-side quality enhancement before prediction. We propose a multi-scale dense residual network (MSDRN) to implement both EPI super-resolution and quality enhancement. Transfer learning strategies are used to improve the training performance of quality enhancement. The experimental results show the superior performance of the proposed framework over existing methods in terms of rate-distortion performance.INDEX TERMS Light field image, sparse compression, EPI super-resolution, quality enhancement, convolutional neural network.

Tensor-Based Light Field Compressed Sensing and Epipolar Plane Images Reconstruction via Deep Learning

“…Chen et al [17] proposed an LF codec with disparityguided sparse coding over a learning perspective shift LF dictionary based on the selected structural key views. Zhao et al [18] proposed to compress key sub-aperture images (SAIs) using standard video encoder, and predicted the nonkey SAIs by the special structure of EPI.…”

“…Yuan et al [37] proposed an EPI enhancement deep CNN to restore the geometric consistency of LF images. Zhao et al [18] proposed a novel LF images sparse compression framework, using EPI superresolution for complementing the information loss caused by sparse sampling.…”

“…But in the second stage, we apply epipolar plane images (EPI) for the deep reconstruction of images. In recent years, the superior performance of EPIs has been used in depth estimation [8], [9] and super-resolution (SR) [18], [37] in LF. To take advantage of EPIs, we use them to solve the deep reconstruction of images and solve the optimal reconstruction problem.…”

Image compression using singular value decomposition by extracting red, green, and blue channel colors

“…As we know, the size of an image is quite large because the image size is depending on the number of pixels. Issues such as storage space and the need for rapid transmission data image through the internet and networks 169 have resulted in the development of a wide range of image compression techniques to reduce the physical size of files [10]. In such cases, image compression techniques are introduced to decrease quantity data of image to signify exactly or approximately the similar information.…”

An effective image compression technique based on burrows wheeler transform with set partitioning in hierarchical trees

“…Also, small learning methods are used to produce a quality system which is ultimately more stable. There is no log board in the JPEG model 11 . The process pressure is mainly exerted on all sources of the studied images.…”