Dense Light Field Coding: A Survey

Abstract

Light Field (LF) imaging is a promising solution for providing more immersive and closer to reality multimedia experiences to end-users with unprecedented creative freedom and flexibility for applications in different areas, such as virtual and augmented reality. Due to the recent technological advances in optics, sensor manufacturing and available transmission bandwidth, as well as the investment of many tech giants in this area, it is expected that soon many LF transmission systems will be available to both consumers and professionals. Recognizing this, novel standardization initiatives have recently emerged in both the Joint Photographic Experts Group (JPEG) and the Moving Picture Experts Group (MPEG), triggering the discussion on the deployment of LF coding solutions to efficiently handle the massive amount of data involved in such systems. Since then, the topic of LF content coding has become a booming research area, attracting the attention of many researchers worldwide. In this context, this paper provides a comprehensive survey of the most relevant LF coding solutions proposed in the literature, focusing on angularly dense LFs. Special attention is placed on a thorough description of the different LF coding methods and on the main concepts related to this relevant area. Moreover, comprehensive insights are presented into open research challenges and future research directions for LF coding. INDEX TERMS Camera array, image compression, light field, plenoptic, video compression.

The JPEG Pleno Light Field Coding Standard 4D-Transform Mode: How to Design an Efficient 4D-Native Codec

“…A wide range of LF coding solutions have been proposed, from well-know, off-the-shelf standard codecs, to those specially designed for LF data. Two surveys on LF field coding are available in [12], [13], which analyze and discuss not only the LF coding solutions themselves but also their associated RD performance. An important aspect to be kept in mind, not directly related to the used technology, but paramount from a market point of view, are the LF codec licensing conditions.…”

“…However, even for moderately small 4D block dimensions, this value is very high. For example, for (t, s, v, u) = (13,13,31,25), the maximum run of zeros would be 4423, and such a value would result in a very high number of possible (run, bit-depth) pairs to encode, which would negatively impact the entropy coding performance. Fortunately, although theoretically possible, runs of zeros as large as these are very unlikely.…”

“…The MuLE-TSR RD performance is assessed using the experimental conditions described in Section VII derived from the JPEG Common Test Conditions (CTC) [11]. For this performance assessment study, the tested 4D-DCT transform dimensions are (t, s, v, u) = (13,13,31,25) and the RD performance is computed for at least one pattern of each of the types of scanning patterns as listed in Table 1, taking care that representative patterns according to the ICE values (see Fig. 6) are chosen, notably:…”

“…The MuLE-TSB RD performance assessment is made under the same JPEG Pleno CTC [11] as before, described in Section VII-A for 4D transform dimensions fixed to (t, s, v, u) = (13,13,31,25). As for MuLE-TSR, the RD performance is first assessed for the same scanning patterns as in Table 2.…”

“…8. Therefore, the MuLE-MTH [11] for MuLE-TH and MuLE-MTH (using transform dimensions (t, s) = (13,13) and (v, u) as specified in second column).…”