Improved Deep Distributed Light Field Coding

Abstract

Light fields enable increasing the degree of realism and immersion of visual experience by capturing a scene with a higher number of dimensions than conventional 2D imaging. On another side, higher dimensionality entails significant storage and transmission overhead compared to traditional video. Conventional coding schemes achieve high coding gains by employing an asymmetric codec design, where the encoder is significantly more complex than the decoder. However, in the case of light fields, the communication and processing among different cameras could be expensive, and the possibility of trading the complexity between the encoder and the decoder becomes a desirable feature. We leverage the distributed source coding paradigm to effectively reduce the encoder's complexity at the cost of increased computation at the decoder side. Specifically, we train two deep neural networks to improve the two most critical parts of a distributed source coding scheme: the prediction of side information and the estimation of the uncertainty in the prediction. Experiments show considerable BD-rate gains, above 59% over HEVC-Intra and 17.45% over our previous method DLFC-I.

Learning-based lossless light field compression

“…Better performance of Cross arrangement can be noted compared to Corner arrangement. These results are also aligned with previous research [32].…”

“…Finally, the rest of the views are predicted from their neighboring four views in the third level. Besides utilizing corner views as reference views, we also examine cross arrangement, which proved to be superior to the corner arrangement in recent work [32]. We note that the corner approach is more flexible in selecting reference view as the cross approach demands odd angular size in order to select reference views.…”

“…First, the view synthesis method for both arrangements, Corner and Cross, are pretrained. Here we used the same settings as done in [32]. Then, the view synthesis network parameters are fixed, and we train the residual estimation network.…”