Shearlet Transform-Based Light Field Compression Under Low Bitrates

Waqas, Ahmad; Vagharshakyan, Suren; Sjöström, Mårten; Gotchev, Atanas; Bregović, Robert; Olsson, Roger

doi:10.1109/tip.2020.2969087

Cited by 32 publications

(28 citation statements)

References 40 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The exploitation of the spatial and temporal correlations in the pseudo-videos (produced by proper PIs arrangement) is made through MV-HEVC, the HEVC multi-view extension [21]. The coding solution proposed in [26] encodes nondensely angular sampled LFs by partitioning the LF data into key views and the so-called decimated views. While the key views are coded using MV-HEVC, the decimated views are predicted using a shearlet transform-based extrapolation, and the generated residual also coded with MV-HEVC.…”

Section: A Most Relevant Light Field Coding Solutionsmentioning

confidence: 99%

The JPEG Pleno Light Field Coding Standard 4D-Transform Mode: How to Design an Efficient 4D-Native Codec

et al. 2020

View full text Add to dashboard Cite

The increasing demand for highly realistic and immersive visual experiences has led to the emergence of richer 3D visual representation models such as light fields, point clouds and meshes. Light fields may be modelled as a 2D array of 2D views, corresponding to a large amount of data, which demands for highly efficient coding solutions. Although static light fields are inherently 4D structures, the coding solutions in the literature mostly employ traditional 2D coding tools associated with techniques such as depth-based image rendering to generate a residual, again to be coded using available 2D coding tools. To address the market needs, JPEG has launched the so-called JPEG Pleno standard, which Part 2 is dedicated to light field coding. The JPEG Pleno Light Field Coding standard includes two coding modes, one based on the 4D-DCT, so-called 4D-Transform mode, and another based on depth-based synthesis, so-called 4D-Prediction. The 4D-Transform coding mode standardizes for the first time a 4D-native light field coding solution where the full light field redundancy across the four dimensions is comprehensively exploited, somehow extending to 4D the 2D coding framework adopted decades ago by the popular JPEG Baseline standard. In this context, this paper describes and analyzes in detail the conceptual and algorithmic design process which has led to the creation of the JPEG Pleno Light Field Coding standard 4D-Transform coding mode. This has happened through a sequence of steps involving technical innovation design and integration where increasingly sophisticated coding tools have been combined and improved to maximize the final rate distortion (RD) performance. INDEX TERMS JPEG Pleno standard, Light field coding, 4D-Transform mode, 4D-DCT.

show abstract

Section: A Most Relevant Light Field Coding Solutionsmentioning

confidence: 99%

The JPEG Pleno Light Field Coding Standard 4D-Transform Mode: How to Design an Efficient 4D-Native Codec

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Narrow baseline LFs, e.g., when the LF is captured by a hand-held camera, lead to low disparity between SAIs. Consequently, several authors have proposed to only encode and transmit some SAIs, normally referred to as structural key views (SKVs), and then transmitting additional information in the bitstream to the decoder to generate the remaining non-SKVs [42]- [48]. These approaches are normally, structurally similar, but the type of additional information varies.…”

Section: Sai-based Related Workmentioning

confidence: 99%

“…In [47], the non-SKVs are encoded using a graph learning approach which estimates the disparity among the views composing the LF. Finally, in [48], the non-SKVs are generated using a shearlet-transform-based prediction scheme which is shown to be efficient when reconstructing densely sampled LFs under low bitrates. Although these approaches are capable of high coding efficiency, their performance is heavily dependent on the SKVs selection.…”

Section: Sai-based Related Workmentioning

confidence: 99%

Light Field Image Coding Based on Hybrid Data Representation

et al. 2020

View full text Add to dashboard Cite

This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring micro-images. The proposed light field coding solution allows flexibility for several types of baselines, by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental results show consistent average bitrate savings when the proposed codec is compared to relevant state-ofthe-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVCbased pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and WaSP, as these codecs are only able to achieve 11% and −14% bitrate savings over the same HEVC-based benchmark, respectively. The performance of the proposed coding approach is also validated for light fields with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVCbased benchmark, as well as MuLE and WaSP.INDEX TERMS Light field representation, light field image coding, HEVC, pseudo-video sequence, spatial pixel prediction, least squares prediction.

show abstract

“…Similarly, the results in [39], [40] also used learning methods to study and improve the quality of light field rendering. Vagharshakyan et al [41], [42] proposed using the shearlet transform to study the sampling and reconstitution of light fields. They used a sparsely represented EPI in a directionally sensitive transform domain obtained from an adapted discrete shearlet transform.…”

Section: Light Field Reconstructionmentioning

confidence: 99%