Multi-Stream Dense View Reconstruction Network for Light Field Image Compression

Liu, Deyang; Huang, Yan; Fang, Yuming; Zuo, Yifan; An, Ping

doi:10.1109/tmm.2022.3175023

“…Concurrently, Yang et al [30]'s method involves sampling images into sparse viewpoint arrays, discerning differences between adjacent microimages, and reconstructing the holistic holographic image leveraging the residual viewpoints and parallax. In a similar vein, Liu et al [31] implemented sparse sampling of dense LF SAIs, transmitting merely the sparse SAIs and employing a multi-stream view reconstruction network (MSVRNet) at the decoder side to reconstruct the dense LF sampling area, yielding commendable outcomes.…”

Section: Related Workmentioning

confidence: 99%

Light-Field Image Compression Based on a Two-Dimensional Prediction Coding Structure

Shao,

Bai,

Jiang

et al. 2024

Information

1

0

View full text Add to dashboard Cite

Light-field images (LFIs) are gaining increased attention within the field of 3D imaging, virtual reality, and digital refocusing, owing to their wealth of spatial and angular information. The escalating volume of LFI data poses challenges in terms of storage and transmission. To address this problem, this paper introduces an MSHPE (most-similar hierarchical prediction encoding) structure based on light-field multi-view images. By systematically exploring the similarities among sub-views, our structure obtains residual views through the subtraction of the encoded view from its corresponding reference view. Regarding the encoding process, this paper implements a new encoding scheme to process all residual views, achieving lossless compression. High-efficiency video coding (HEVC) is applied to encode select residual views, thereby achieving lossy compression. Furthermore, the introduced structure is conceptualized as a layered coding scheme, enabling progressive transmission and showing good random access performance. Experimental results demonstrate the superior compression performance attained by encoding residual views according to the proposed structure, outperforming alternative structures. Notably, when HEVC is employed for encoding residual views, significant bit savings are observed compared to the direct encoding of original views. The final restored view presents better detail quality, reinforcing the effectiveness of this approach.

show abstract

“…Subsequently, this work was improved in 30 by adding a multi-view quality enhancement network to ensure the reconstruction qualities of synthesized SAIs. Liu et al 31 constructed a multi-disparity geometry structure of sparse SAIs at the decoder side and then put forward a multi-stream view reconstruction network to reconstruct the entire LF. By exploring the abundant LF geometric structure information, this method achieved a high compression performance.…”

Section: Related Workmentioning

confidence: 99%

“…While the SAI array based methods [18][19][20][21][22][23][24] intent to enhance the compression performance by eliminating redundancies of adjacent SAIs. Wherein, based on the wide applications of Convolutional Neural Network (CNN) in LF image processing, learning based LF view reconstruction methods [25][26][27][28][29][30][31] are introduced into the LF compression. The main idea of learning based LF compression method is to encode sparsely-sampled LF SAIs at the encoder side and synthesize the rest of SAIs with learning based view reconstruction at the decoder side.…”

mentioning

confidence: 99%

Geometry-aware view reconstruction network for light field image compression

Zhang

¹

,

Wan

²

,

Mao

³

et al. 2022

Sci Rep

Self Cite

1

0

View full text Add to dashboard Cite

Light Field (LF) imaging empowers many attractive applications by simultaneously recording spatial and angular information of light rays. In order to meet the challenges of LF storage and transmission, many view reconstruction-based LF compression methods are put forward. However, occlusion issue and under-exploitation of LF rich structure information limit the view reconstruction qualities, which further influence LF compression efficiency. In order to alleviate these problems, in this paper, we propose a geometry-aware view reconstruction network for LF compression. In our method, only sparsely-sampled LF views are encoded, which are further used as priors to reconstruct the un-sampled LF views at the decoder side. The proposed reconstruction process contains two stages including geometry-aware reconstruction and texture refinement. The geometry-aware reconstruction stage utilizes a multi-stream framework, which can fully explore LF spatial-angular, location and geometry information. The texture refinement stage can adequately fuse such rich LF information to further improve LF reconstruction quality. Comprehensive experimental results validate the superiority of the proposed method. The rate-distortion performance and the perceptual quality of reconstructed views further demonstrate that the proposed method can save more bitrate while increasing LF reconstruction quality.

show abstract

“…While the SAI array based methods [18]- [24] intent to enhance the compression performance by eliminating redundancies of adjacent SAIs. Wherein, based on the wide applications of Convolutional Neural Network (CNN) in LF image processing, learning based LF view reconstruction methods [25]- [31] are introduced into the LF compression. The main idea of learning based LF compression method is to encode sparsely-sampled LF SAIs at the encoder side and synthesize the rest of SAIs with learning based view reconstruction at the decoder side.…”

Section: Introductionmentioning

confidence: 99%

Geometry-Aware View ReconstructionNetwork for Light Field ImageCompression

Zhang

¹

,

Wan

²

,

Mao

³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Light Field (LF) imaging empowers many attractive applications by simultaneously recording spatial and angular information of light rays. In order to meet the challenges of LF storage and transmission, many view reconstruction-based LF compression methods are put forward. However, occlusion issue and under-exploitation of LF rich structure information limit the view reconstruction qualities, which further influence LF compression efficiency. In order to alleviate these problems, in this paper, we propose a geometry-aware view reconstruction network for LF compression. In our method, only sparsely-sampled LF views are encoded, which are further used as priors to reconstruct the un-sampled LF views at the decoder side. The proposed reconstruction process contains two stages including geometry-aware reconstruction and texture refinement. The geometry-aware reconstruction stage utilizes a multi-stream framework, which can fully explore LF spatial-angular, location and geometry information. The texture refinement stage can adequately fuse such rich LF information to further improve LF reconstruction quality. Comprehensive experimental results validate the superiority of the proposed method. The rate-distortion performance and the perceptual quality of reconstructed views further demonstrate that the proposed method can save more bitrate while increasing LF reconstruction quality.

show abstract

Multi-Stream Dense View Reconstruction Network for Light Field Image Compression

Cited by 10 publications

References 51 publications

Light-Field Image Compression Based on a Two-Dimensional Prediction Coding Structure

Light-Field Image Compression Based on a Two-Dimensional Prediction Coding Structure

Geometry-aware view reconstruction network for light field image compression

Geometry-Aware View ReconstructionNetwork for Light Field ImageCompression

Contact Info

Product

Resources

About