View Synthesis for Light Field Coding Using Depth Estimation

An, Ping; Shan, Liang; Ma, Ran; Shen, Liquan

doi:10.1109/icme.2018.8486515

Cited by 16 publications

(20 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A prototype is proposed in [18], in which the depth map is computed only by horizontal EPI and optimized through crude error pixel repairing, so it generates the unsatisfactory depth map. To obtain a more accurate depth map, the depth estimation of the structure has been improved in [19]. Although the number of SAIs selected and coded is small, an intermediate SAI can still be approximated once the depth map can be precisely generated, which can contribute to low computational complexity and bitrate.…”

Section: Related Workmentioning

confidence: 99%

“…Although the number of SAIs selected and coded is small, an intermediate SAI can still be approximated once the depth map can be precisely generated, which can contribute to low computational complexity and bitrate. Note that this article is based on our earlier work, as described in [18,19], and the novelties of this article are generalized as follows.…”

Section: Related Workmentioning

confidence: 99%

“…Many technological challenges remain associated with the huge amount of data. To this end, we previously proposed an LF compression method in [18,19]. This article, based on our earlier work, deeply analyses all combinations of depth computation and sub-aperture images sparsely sampling steps to compensate the computational complexity and the coding efficiency.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Light-field compression using a pair of steps and depth estimation

An¹,

Cao²,

Liu³

et al. 2019

Opt. Express

Self Cite

View full text Add to dashboard Cite

Advanced handheld plenoptic cameras are being rapidly developed to capture information about light fields (LFs) from the 3D world. Rich LF data can be used to develop dense sub-aperture images (SAIs) that can provide a more immersive experience for users. Unlike conventional 2D images, 4D SAIs contain both the positional and directional information of light rays; the practical applications of handheld plenoptic cameras are limited by the huge volume of data required to capture this information. Therefore, an efficient LF compression method is vital for further application of the cameras. To this end, the pair of steps and depth estimation (PoS&DE) method is proposed in this paper, and the multiview video and depth (MVD) coding structure is used to relieve the LF coding burden. More specifically, a precise depth-estimation approach is presented for SAIs based on the cost function, and an SAI-guided depth optimization algorithm is designed to refine the initial depth map based on pixel variation tendency. Meanwhile, to reduce running time, intermediate SAI synthesis quality and coding bitrates, including the key SAIs selected and cost-computation steps, are set via extensive statistical experiments. In this way, only a limited number of optimally selected SAIs and their corresponding depth maps must be encoded. The experimental results demonstrate that our proposed LF compression solution using PoS&DE can obtain a satisfied coding performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Light-field compression using a pair of steps and depth estimation

An¹,

Cao²,

Liu³

et al. 2019

Opt. Express

Self Cite

View full text Add to dashboard Cite

show abstract

“…We used the method from [29] to extract the depth map feature of LFI. All SAIs of an LFI are utilized to estimate the depth map according to EPI image analysis.…”

Section: ) Depth Mapmentioning

confidence: 99%

A No-Reference Image Quality Assessment Metric by Multiple Characteristics of Light Field Images

Shan

Meng

et al. 2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

Evaluation of light field image (LFI), especially micro-lens camera light field (LF), is a new and challenging work. The development of image quality assessment (IQA) metric of LFIs relies on the subjective quality assessment database. In this paper, we establish a perceptual quality assessment dataset consisting of 240 distorted images from 8 source images with five distortion types. Furthermore, a no-reference IQA metric is proposed by combining 2D and 3D characteristics of LFI with the Support Vector Regression (SVR) model. The performance of the proposed metric is demonstrated by comparing with some classical full reference IQA metrics both on the presented dataset and a third-party dataset. The experiment results show that our method has a better performance than others.INDEX TERMS Light field images, subjective quality assessment, objective quality assessment, light field characteristics, SVR.

show abstract

“…The unselected SAIs are synthesized by depth image-based rendering technique on the decoder side. Meanwhile, they designed a pixel-variationtendency-based weighted filter to achieve depth optimization [35]. Rizkallah et al [36] encoded the residuals between the synthesized and original views using graph transforms.…”

Section: Related Work a Lfi Compressionmentioning

confidence: 99%

Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Zhao

Yang

et al. 2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

Because of the capacity of capturing both the spatial and angular information of the light rays simultaneously, light field images (LFIs) contain richer scene information compared with conventional images, but at the cost of huge volume. This paper proposes a novel LFI sparse compression framework driven by convolutional neural network (CNN). The epipolar plane image (EPI) super-resolution is for compensating the information loss caused by sparse sampling and the decoder-side sub-aperture images (SAIs) quality enhancement is for compensating the information loss caused by lossy compression. Specifically, we choose those SAIs both in odd rows and odd columns as our key SAIs and compress them using standard video encoder. For those non-key SAIs, we predict them using decompressed key SAIs by taking advantage of the special structure of EPI. The low-resolution EPIs generated from the sparse SAIs are super-resolved by a CNN and the outputs, high-resolution EPIs, are used to rebuild the dense SAIs. Moreover, in order to improve the quality of the predicted SAI, we add decoder-side quality enhancement before prediction. We propose a multi-scale dense residual network (MSDRN) to implement both EPI super-resolution and quality enhancement. Transfer learning strategies are used to improve the training performance of quality enhancement. The experimental results show the superior performance of the proposed framework over existing methods in terms of rate-distortion performance.INDEX TERMS Light field image, sparse compression, EPI super-resolution, quality enhancement, convolutional neural network.

show abstract

View Synthesis for Light Field Coding Using Depth Estimation

Cited by 16 publications

References 22 publications

Light-field compression using a pair of steps and depth estimation

Light-field compression using a pair of steps and depth estimation

A No-Reference Image Quality Assessment Metric by Multiple Characteristics of Light Field Images

Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Contact Info

Product

Resources

About