Light field reconstruction in angular domain with multi-models fusion through representation alternate convolution

Abstract

To alleviate the spatial-angular trade-off in sampled light fields (LFs), LF super-resolution (SR) has been studied. Most of the current LFSR methods only concern limited relations in LFs, which leads to the insufficient exploitation of the multi-dimensional information. To address this issue, we present a multi-models fusion framework for LFSR in this paper. Models embodying LF from distinct aspects are integrated to constitute the fusion framework. Therefore, the number and the arrangement of these models together with the depth of each model determine the performance of the framework; we make the comprehensive analysis on these factors to reach the best SR result. However, models in the framework are isolated to each other as the unique inputs are required. To tackle this issue, the representation alternate convolution (RAC) is introduced. As the fusion is conducted successfully through the RAC, the multi-dimensional information in LFs is fully exploited. Experimental results demonstrate that our method achieves superior performance against state-of-the-art techniques quantitatively and qualitatively.