Light Field Inpainting Propagation via Low Rank Matrix Completion

Abstract

Building up on the advances in low rank matrix completion, this article presents a novel method for propagating the inpainting of the central view of a light field to all the other views. After generating a set of warped versions of the inpainted central view with random homographies, both the original light field views and the warped ones are vectorized and concatenated into a matrix. Because of the redundancy between the views, the matrix satisfies a low rank assumption enabling us to fill the region to inpaint with low rank matrix completion. To this end, a new matrix completion algorithm, better suited to the inpainting application than existing methods, is also developed in this paper. In its simple form, our method does not require any depth prior, unlike most existing light field inpainting algorithms. The method has then been extended to better handle the case where the area to inpaint contains depth discontinuities. In this case, a segmentation map of the different depth layers of the inpainted central view is required. This information is used to warp the depth layers with different homographies. Our experiments with natural light fields captured with plenoptic cameras demonstrate the robustness of the low rank approach to noisy data as well as large color and illumination variations between the views of the light field.

Fast Light Field Inpainting Propagation Using Angular Warping and Color-Guided Disparity Interpolation

“…For comparison [8] faster than proposed in [7], and ∼ 2.2 times than [8]. Regarding the later, our method performs faster for larger inpainting masks.…”

“…Because the mask applied on light field Bee2 has small size, method of [8] is faster. Indeed, the singular value decomposition used has a O(N 4 M ) complexity linearily dependent on mask's size, whereas our method has O(P ) complexity in this case.…”

“…See misalignment of the edges of the central leave. c,g) With method proposed by [8] as reference result. d,h) Using our method which result matches reference.…”

“…However, because of the greedy nature of the algorithm and the fact that the views are processed separately, the inpainting results may contain angular inconsistencies. The authors in [7] exploit smoothness priors in the epipolar plane images (EPI) to coherently propagate information across views using PDE-based diffusion, while a method based on matrix completion is proposed in [8], assuming a low rank prior for the light field data.…”

Light Field Saliency Detection With Deep Convolutional Networks

“…From the table, we can see that the network using 9 × 9 viewpoints shows the overall best performance. Increasing the angular resolution to 11 × 11 cannot improve the performance, which can be explained by the fact that the viewing angles at the boundary are very oblique [55] and the narrow baseline of the light field camera leads to high viewing redundancy with higher angular resolutions [7], [56]. 4) Overfitting issues: Overfitting is a common problem related to training a CNN with limited data.…”